 Hi, I'm Zor. Welcome to Nezor Education. Today we will talk about sequence limits, and primarily what we are interested in is not just that the sequence has a limit and basically goes, converges to this limit, but the question is how fast it happens, because there are certain sequences which are approaching their limits faster than others, and our purpose is basically to measure the relative speed of converging to the limit. This lecture is part of the advanced course of mathematics for high school students and teenagers on Unizor.com. I suggest you to watch this lecture from this website, because it has lots of other educational functionality. The site is free. So, talking about speed or rate of approaching the limit, we will consider two cases, when the limit is zero and when the limit is infinity. If the limit is a constant which is not zero, then basically if xn goes to a, then xn minus a is infinitesimal variable, which goes to zero and we will basically measure the speed of approaching zero of this particular sequence. So, we can always talk about either infinitesimal sequence or the sequence which is infinitely growing. And let's assume for simplicity that we're talking about positive case only. So, all the xn's, at least starting from certain n, are positive and this a is either zero or positive infinity. And again, our purpose is to compare the speed of going, approaching that particular limit. But from a couple of examples. Okay, example number one, you have a sequence of n natural numbers and then you have a sequence of n plus 1 natural numbers. Both are going to infinity as n goes to infinity. Question is, which one is faster and how can we compare actually the speed of this type of approaching the infinity. It's not easy to compare. Well, there is actually a very simple way to do it. We can always measure what's the proportional difference between these. If this proportional difference is growing, then one of them is going faster. Basically, the proportional difference is difference divided by at least the first one, let's say. So, it's n plus 1 minus n divided by n, which is 1n. Now, as n goes to infinity, this proportional difference obviously is going to zero. Which basically tells us that, well, there is probably not such a big difference between growing to infinity of this sequence versus this. The speed is basically very, very close. And as a matter of fact, as n grows, then the difference, the proportional difference between these two sequences is basically getting negligible. So, that's one example. Let's consider another example. One is n, another is 2n. Well, let's do exactly the same thing. What's the difference between them? Well, it's 2n minus n divided by one of them to get proportional difference, which gives us 100%. Now, this is a constant. It's independent of n, which means that this relative difference is constant. Which probably means, if you can translate it into regular language, that these two sequences are growing at the same rate. And yes, indeed, if n goes twice as big, this goes twice as big and this goes twice as big, right? So, that's what actually makes us to say that these two sequences have the same rate of increase as n goes to infinity. And then I have the third example, natural logarithm of n. Well, this thing grows slower. Remember the graphs? Graphs of y is equal to x. Basically, if x is our number, it's this line, right? And logarithm is this line. It's growing to infinity, but it's changing its speed. It seems to be less or less, and this speed seems to be the same, right? So, there is some kind of a notion that this is supposed to go to infinity faster than this one. Well, let's try just to have a couple of examples and let's see if that actually happens. We will go into the proofs a little later, but right now I just want to have examples. So, if n is 10, my logarithm is 2.3, and the difference between relative proportional difference between them is what's like 7.6, whatever, and that's 77% of the 10, right? Now, if n is 1000, the natural logarithm is 6.9 something, which is greater than 99%. So, it's growing, and if you will do a little bit more precise calculations and go to even further, you will get the greater and greater number here. So, the difference, the proportional difference between them is growing, which basically tells us that most likely it seems to be that this is growing much faster than this one. So, the rate of change can be measured in this type of calculations. Next, let's consider instead of going to infinity, let's go to 0, infinitesimals, alright? So, we will compare the sequence 1n and the sequence of 10n. The situation is kind of similar to we had n and 2n. So, let's just again measure the proportional difference. Proportional difference is what? 9n, right? So, this is 1n, this is 10n, the difference is 9n. Well, plus or minus doesn't really matter. For instance, it's a minus, if you subtract this from this, but it doesn't really matter. This from this goes to 9n, divided by this would give us just 9, which is 900%, right? So, the relative difference between them, the proportional difference is always constant, which means that these two guys are going to 0 with the same speed. They are infinitesimals of the same rate of diminishing, okay? Let's have another one over natural logarithm of n. Now, from the previous, when we were talking about infinity, n was growing significantly faster. Rate was greater than logarithm n. So, we are right now inverting that thing. So, it looks like this should go to, since the denominator grows faster, then the fraction should go to 0 faster than this one. And I have some statistics. If n is 10, then this is one-tenths. And this is, well, actually my example probably would be better if I would take decimal logarithm. So, if n is 10, then it's one over one, which is one. And the difference is 9-tenths, which is 900%, right? 9-tenths divided by one-tenths. But if you have 1,000, no, actually 100 even here. I have 100 here. So, here we have, oh, you know what? I do have actual numbers for natural logarithm. I don't have to do this. So, let's do natural logarithm. I like it much better. So, natural logarithm is 0.4343, which represents the difference is, so this is 0.1, this is 0.4343. So, the difference is 0.3-something divided by 0-1, so it's like 3-something. So, it's greater than 300%. And if I have 100, then this is 100, and this one is 0.2174. So, if this is 100, and this is this, so the difference is 20 divided by 0.1, so it's 20 actual, so it's 2,000, more than 2,000%. So, as you see the difference, as we measure it, the proportional difference is growing. And again, I'm not proving anything, but at least just looking at this, it looks like this thing is going to 0 faster. Okay, that's basically it for examples. I think it's time now to do the definitions of whatever we are using for rates of change. So, basically, mathematicians decided to come up with two different concepts. They are called big O and little O. Well, big O is like a capital O, and little O is a lower case, O. Now, the big O means the same speed, the same rate of change, whether it's to infinity or to 0. And the little O means that something which is on the left would be smaller than something on the right. And here is basically the more or less exact definition. So, if you have two sequences, Xn and Yn, and the ratio, let's consider the ratio, and obviously we assume that Yn is not equal to 0. So, this ratio represents basically, well, almost the same as whatever the relative proportional difference which we were calculating because what we did was Yn minus Xn divided by Xn, right? Which is the same thing as Y over Xn minus 1, right? So, it doesn't really matter. This constant is irrelevant. So, what I would like to say is the following. So, if these two are going to infinity, Xn and Yn, but this ratio is within certain limits which are positive. We are talking about positive members of the sequence and A and B are supposed to be positive. That's very important that A is greater than 0. So, everything is greater than 0. So, if this is within these boundaries between A and B, both are positive finite numbers, then we can say that they are growing with the same speed, and we can say that the rate of change of Xn is the big O of the rate of change of Yn. That's how big O is defined. This is the definition of the big O. If, however, we can say that this thing is infinitesimal, when Xn and Yn are going to infinity, this thing is infinitesimal. What does it mean, basically? Well, it means that Yn is growing faster. So, the denominator grows significantly faster than the denominator, and then we can say that Xn is equal to lowercase O, which is little O of Yn. So, big O means the same. Little O means that the left part is smaller in some sense than this one. But what actually we mean is that the rate of growth is smaller. Now, let's talk about both of them, Xn and Yn, instead of going to infinity, they're both infinitesimals. Same thing. If their ratio is between A and B, and A is strictly greater than 0, so basically it's a positive constant, if this ratio is within these two boundaries, then we can say that Xn is equal big O of Yn. So, for infinitesimals is exactly the same. And obviously it's the same for little O. If this is infinitesimal, then Xn is little O of Yn. So, basically we have exactly the same definition for sequences going to infinity, and sequences going to 0, as infinitesimals. Infinite growing and infinitely diminishing. If the ratio between them is within certain boundaries, which definitely is supposed to be greater than 0, that's very, very important, then we can say that they are growing with the same speed. The rate of change is the same. And if it's infinitesimal, their ratio, then obviously we are saying it's a little O. So, little O, big O notation is how we are talking about the rate of change. Either it's infinitely growing or infinitely diminishing to 0. Now, I was talking about positive sequences, but you understand that if it's negative, then we have to just change some signs, but basically it's exactly the same. Now, as far as practical matters, what's very important is, well, before this was actually purely theoretical kind of talking about different sequences. But nowadays, with computers coming in and basically trying to do some manipulations with huge amount of data, this thing is very important to analyze algorithms. Because if the algorithm is with the increase of N, number of objects that you are actually analyzing, if it's growing like N, let's say, with some factor, then it's one thing. If it's going like N square, for instance, which is significantly faster, that's another thing. With the small amounts of data, it's probably not very significant because computers are fast. But with N, number of objects which we are analyzing, growing significantly to astronomical numbers, that becomes actually important. And computers should really go through algorithms which provide the lowest possible algorithmical complexity, if you wish. So these things are very, very important. And if you will ever go to interview to Google, for instance, you will be asked to evaluate the rate of growth. Usually, it's with infinitely growing sequences. So you will be asked to evaluate the rate of growth. Is it growth like N or is it growth like N factorial or is it growth like logarithm N, where N is number of objects, which algorithm is analyzing. All right, now it's time to go through just a couple of examples to illustrate what is exactly big O and little O things. Now from our previous examples, for instance, you remember that N and 2N, the relative difference was basically constant between them. And if you would take the ratio XN over YN, you will have what, 1 half constant. Well, constant means it's definitely bounded and boundaries are greater than 0. So obviously this is big O of this. But in case of N and natural logarithm, and you cannot say this, because this is much greater, this is faster growing. So the right way to say is that this is little O of that, because this is growing much slower. So it's smaller, basically. Little O means smaller in both cases, infinitely growing and infinitesimals. All right, now time to go through these examples with proof, right? So my first example is XN is equal to 2N squared plus N minus 1. YN is equal to N squared minus 1. Well, this is easy. Whenever you have polynomials, it's usually very simple thing. And one of the previous lecture actually was dedicated to ratio of two polynomials. And we have basically agreed that the most important thing is, forget about all other members of the polynomial except the one in the greatest power, which is 2 and 2 here, which means basically they should be of the same rate of change. And it's just the coefficients between these two, which are important. Now, let me just do this according to... So this is XN and this is YN. And let me just do this ratio and let's analyze it how it behaves. So it's 2N squared plus N minus 1 divided by N squared minus 1. This is 2N squared minus 2 plus N plus 1, right? Divided by N squared minus 1, which is... This divided by this is 2 and this divided by this is N minus 1 times N plus 1. So it will do this. And as we see, this ratio is always between 2 and about 3, actually, right? So our boundaries, A and B boundaries are 2 and 3. This ratio is always within these two boundaries, which means that XN is big O of YN. So that's my first example. That's easy. Yeah, right. With polynomials, it's always easy. Now, my second example is also easy. My second example is XN is equal to 1 over N, so it's infinitesimals. And YN is equal to N divided by N squared minus 1. Okay. Well, again, in these cases, we are dealing with not just polynomials but ratio between two polynomials. And whenever polynomials are involved, again, you pay attention only to the highest power. So basically, this is N to the first power. This is N to the second power. And you can assume it would be like 1 over N, basically. Like if you will completely forget all these tails, then you would just reduce it by N. It would be 1 over N, and this is 1 over N. It's supposed to be the same, more or less, right? Well, let's just do it a little bit more precisely. Equals to 1 over N divided by N times N squared minus 1. So this is N squared. Let's divide. It's 1 minus 1 over N squared. And obviously, we see that as N is increasing, so we have upper bound 1. But lower bound can be, well, let's say if N is equal to 2, it's 1 minus 1 quarter, which is 0.75. But if N grows, it will be even closer. So something like 0.75 is holding for all N. And that's fine. That means that, again, XN is big O of YN. Both are infinitesimals of the same rate of diminishing. Let's go on. Now, the third example is a little bit more complex. So the first two examples were about big O, now about little O. So my first example is logarithm N and N. So what we are looking is this ratio. Now, I would like to prove that this ratio is infinitesimal as N goes to infinity. Okay, I mean, there are many different ways of doing it. I just suggest this one, which basically doesn't really depend on certain more complex theories or some prior knowledge or whatever. So here is the way how I suggest to do it. First of all, this is equal to 1N logarithm N, which is logarithm of N to power 1N, right? Remember, if you have something in power and you have a logarithm, that power actually goes like a factor. So, instead of proving that this is going to 0, I will prove that N to the power 1N goes to 1. That's the same thing, right? Because logarithm of 1 is equal to 0, so if this is going to 1, then logarithm of this is going to... If this goes to 1, logarithm goes to 0. So this is what I'm going to prove. Or, which is exactly the same thing, I will prove that N to the power 1N minus 1 is going to 0. That's, again, the same thing. So I just equivalently transformed whatever I have to prove. So I'm going to prove that this is infinitesimal sequence. Okay, let's call it ZN, right? Now, I'm going to use the Newton's binomial formula. And here is how I would like to do it. First of all, what is 1 plus ZN to the power of N? Well, if ZN is 1 plus N to the power 1N minus 1 to the power of N, N to the power 1N to the power N, right? And this is N. So remember, when you have something in power and then you raise it into another power, you just multiply the powers. So 1N multiplies by N, so you have just plain N. On another hand, this can be represented as Newton's binomial. Well, just in case, if you forget about it, let me just write the formula. A plus B to the power of N, sum of I from 0 to N, C and I, A to the power of N minus I, B to the power of I. Okay? All right, so A is equal to 1. So in my case, I have this type of thing. So A to any power, 1 in any power will be 1 anyway. So let me just write it closer. So that's what I have. All right. So for I is equal to 0, I have C of N by I, but by 0, which is 1. Times B to the power 0, which is 1. Okay, plus. Next, the number of combinations from N to 1 is N, times B to the first degree. Well, in this case, it's not B, it's ZN. Next, N, N minus 1 divided by 1.2, that's number of combinations from N by 2, times ZN square. Plus blah, blah, blah. Now, all this blah, blah, blah is positive. So what I can say is that 1 plus ZN to the power of N is greater. I can put greater or equal, it doesn't really matter. Then I will just choose only one member from here and drop everything else. Obviously, I will diminish it, right? That's why it's less. So it's N, N minus 1 divided by 2, ZN square. But what is on the left? That's N. So what kind of inequality we have right now? We have N greater or equal N, N minus 1 to ZN square. Or 1 greater than N minus 1 over 2, ZN square. Or ZN square less than or equal 2 over N minus 1. So now, we see that as N goes to infinity, the right thing goes to zero. It's an infinitesimal, right? Which means that this one is also infinitesimal. It goes to zero and that's exactly what we have to prove. That ZN goes to zero. Which means N to the power of 1N goes to 1. And which means that logarithm of this goes to zero. And what exactly, that's exactly what we have to prove. So it's a little longer. And then there is this little trick with Z. Well, look, I mean, it's better to have some kind of a trick than to rely on certain theories which you know or you don't know, remember or don't remember. At least it's all straightforward here and everything is presented. Well, except maybe you might not remember Newton's binomial formula, but this is also very well presented in one of the lectures on mathematical induction. In this course. Alright, so that's my example number three. And the last one, that's easy, but it's also using binomial. Okay, 1 over 2 to the power of N. I didn't really conclude this. Let me just conclude that logarithm N is little o of N. Basically that was the result of whatever I did in the previous problem. Now, now I'm comparing two infinitesimals. 1 over 2 to the power of N and 1N. Well, again, you probably feel that this thing goes to zero much faster. It's one half, one quarter, one eighths, one sixteenths. And this is one two, one second, one third, one fourth, one. It seems to be faster. Question is, is it really faster? Is this one equal to little o of this one? Which means, is this ratio an infinitesimal? Which is what? N over 2 to the power of N, right? Okay, let's just think about it. I will use the same binomial formula. A plus B to the power of N is equal to sigma from I from zero to N, C and I, A and minus I, B, I. For A is equal to, B is equal to 1. So 1 plus 1. 2 to the power of N is equal to sigma, C, N, I, I from zero to N. A to the power, B to the power, since they are 1, I just estimate. So I will use this formula. So what do I have here? I have N over, from N by zero, number of combinations is 1 plus N. N, that's number of combinations from N by 1. N square, sorry, N minus 1 by 2 plus, etc. Now, I will do exactly the same as in the previous problem. If I diminish my denominator, I will increase the fraction, right? So I will use only this one. And that's why I have this inequality. So since this denominator is smaller than this one, the whole fraction is greater than the whole fraction here. And this is equal to 2 over N minus 1. And this is infinitesimal, as N goes to infinity, which means that 1 divided by 2 to the power of N is little o of 1 over N. So again, binomial formula is used here. It's always handy to evaluate the relative growth. You know, use the binomial formula in some way. Okay, these are a few examples and obviously there are many problems related to this big old little old thing, which probably I will try to present as exams maybe or something like self-study. Other than that, I do suggest you to read the notes for this lecture because every lecture on Unizor.com has very detailed notes. Just to make sure that you understand everything. Other than that, that's it. Thank you very much and good luck.