 Hello and welcome to the next lecture in the course on introduction to computer and network performance analysis using queuing systems. I am Varshaapte. I am a faculty member in the department of computer science and engineering IIT Bombay. So today we will talk about Little's law. So this is again the usual recap of all the notation. You can just refer to this in the slides. Remember that we are after all of these metrics, performance metrics of a standard queuing system. And I want to take that opportunity to remind everybody that all of these metrics were actually averages. These are averages and those averages are also over a long period of time t. Remember we had done some observational laws and at that time we had assumed some time t and that time t itself was supposed to be long enough so that some sort of startup conditions and end conditions can be ignored. So that kind of a view of things where we look at system when there are not any disturbances kind of going on in it is called steady state. So we look at a system when the averages in the system are not changing. So this is a reminder because what we are going to look at today which is Little's law is applicable only as steady state. So remember this word steady state. This is when the system is not changing much anymore. The arrival rate is some at some stable rate, the service rates are stable, the number of servers of course do not change, buffer size does not change. These kind of parameters of the system do not change and there are not also effects that you will see at the start and at the end. So just when you are observing the system for over a long time that is what steady state means. This is again a recap of everything that we know about queuing system so far. Low load asymptotes for any queuing system are easy to find. High load asymptotes are also actually easy. We can write for GGCK which is the most general queuing system that we have learned but when there is just some general lambda greater than 0 which cannot be assumed to be very small or very large, then we know that actually for finite buffer we can almost not calculate anything. For infinite buffer we still have throughput and utilization both finite buffer and infinite buffer. We do not have these 4 metrics. We do not know how to calculate the number in queue, the number in system, waiting time, response time. We do not know this for both finite and infinite buffer. And in last class I told you that we need something called stochastic models and stochastic processes actually. It is called stochastic processes to do this but we are not yet going to do so much advanced mathematics. We are going to continue and try to use laws and rules and that is a little more intuitive. So, in that journey the next thing to learn is Little's law which basically does not actually directly allow us to calculate each of this but it gives us a relation between those. So, we will come to that in a few slides. First what I want everybody to think is if we had to, if you had to just reason about how a response time and number in the system related. I think you would be able to come up with some intuition. In a way what I am going to do in these next few slides is that actually intuition does not work that well in this particular case. So, what do I mean? I want us all to do one thing that I am calling a thought experiment. Let us just do an imagination. What do I want you to imagine? Suppose you have to meet a friend at a ticket booth. Let us say it could be a movie or some other event that you are trying to buy a ticket for and that is going to be at some ticket booth and you are going to go there with a friend to buy the tickets. You are just standing away from the booth. You will join the line only when the friend comes. You do not want to join the line unless the friend comes because you do not even know whether the friend is coming. You have some other things to do. So, you are standing away from the booth. Now, suppose your friend is getting very late. So, you just standing near the booth for a long time and what that allows you to do is it allows you to just observe the booth. Just allows you to observe what is going on with the arrivals and departures of people at the booth. So, what do you note? You are, you know, let us say you are learning this course. So, you feel like noting some metrics. So, you just keep track of what is the average number of people standing in the queue and in this case, this will include the one buying the ticket. So, this will be like buffer plus service because one person who is next to the window will be the one who is actually buying the ticket. So, suppose that whole thing is 10. Now, and you also kept track of how long does it seem to be taking to buy the ticket, one person how long is it taking to full amount of time to buy the ticket and you noted that that average is 5. So, you can say that this is basically your tau. So, total 9 plus 1 we have 10 people that can stand out of whom one will be in service. So, it will be like a gg19 system with tau is equal to 5 minutes. Now, if you have done all of this observation, what is your estimate of the expected time from the time your friend arrives which is when you will join the queue to the time that you leave from the booth with the ticket. Hopefully, it will not close, the ticket booth will not close. We are assuming steady state nothing changes about the system. We assume steady state. So, given that all of these observations are just what are remaining for the whole observation time and for long time then what is your estimate. So, just I will draw this as a picture here. So, this is the queue, this time let us draw people and this is let us say the ticket booth. There is of course, the person who is selling the tickets and there will be one person here at the front of the booth who is actually buying the ticket. So, including this person, there is this one person here and then rest of them we know on an average 9 plus 1 is equal to 10 is what you have seen and this thing here takes 5 minutes. Now, you are trying to estimate what will happen when your friend comes, you and your friend, you are standing somewhere here far away. Now, suppose your friend also comes and you go and stand in line. Of course, now we are going to show you as just one person because you are not going to stand behind one another. It is just one position that you will occupy. Your friend will also be with you but you will occupy one position. So, you will stand at the end here. So, what does your intuition tell me, tell you. If I did not had never done this course and I was just thinking myself, I would have done the calculation that there are around 10 people in this queue and let us say in the worst case when I join the queue all of the full 5 minutes of this person are left. So, to clear this all these people in front of me it will take 10 into 5 equal to 50 minutes and then I have my own time which is 5 minutes. So, I could take 55 minutes, that would be my expectation or luckily if I join the queue when just this person is leaving. So, or only a few seconds are left. So, then I do not have to, then it is like I am the 10th person here and there are only 9 in front of me. So, it will be 9 multiplied by 5 whose service has just started and then plus my 5 and this will give me 50 minutes. So, maybe I will estimate anything from 50 to 55 minutes as my expected time that I have to stand in the queue until I can get a ticket. Now, this is I am just sort of summarizing all that what I just said observed average in the system is 10, you become the 11th and then you count 10 full service times and then the one remaining service time. So, you will get a 50 to 55 minutes range. This is what you will do if you have not done this course. Now, in general so, can we say that R should be related in some way to the number in system and tau you know as going from n tau to n plus 1 to tau. This reasoning seems quite reasonable. I mean what is what is it you know if you again if you do not remember some nuances that you will learn today this does not seem very wrong. So, question is is this what little slot does? Is it somehow relating the average response time with the average number in system and service time? Because that seems to be what is what is obviously what you should do seems like this is obvious. Turns out very surprisingly it does not do that. This is not the relationship that little slot is about. It relates n and R, but not through the service time. Actually it relates R and n through inter-departure time which in other words is 1 over throughput. So, n and R is related through lambda by little's law capital lambda. So, let us look at what is little's law. So, I just wanted to tell this whole story to draw the moral of the story that little's law is actually not very intuitive. If you try to do some direct reasoning it turns out to be wrong and there are some explanations, intuitive explanations of little's law, but they are not very straightforward, but the law is has been proved mathematically. The law is correct of course it is not wrong and this is what it is. So, for little's law what we do is we start with a very very general definition. We do not start with our standard queue. We start with a general definition of a of some region. There is some region in which customers arrive and they depart. And inside the region there can be anything. There can be sort of you this can for example be a packet network with some connections. It can be some other, it can be some multi-tier like web applications or something where I have some server here, some and which actually you go to the web server, then you go to the database server. This could be like a website where there are multiple servers or this could be representing hardware where I have CPU, I have disk. So, it can be anything. It also need not be an entire system. It can be a part of the system. So, this in here I showed that there can be multiple systems, but that is not the case. The region can be just focused on a part of the system. For example, if there was only one queue, I could say that this the little's law region is only the buffer and the actual server is outside the region. This is also possible. So, the little's law region is actually very flexible. It just depends on what you decide to draw a boundary on as long as it satisfies some conditions. So, it is a well defined region through which customers arrive and depart. It can have multiple service stations or even part of a service station as long as everything inside is what is called work conserving. What does work conserving mean? It means that whatever the servers are inside the system, they will not be idle if there is any customer in their in its queue. So, we can never have a situation that there are customers here, but the server is just not serving them. This situation cannot happen. The other condition is that customers are not created or destroyed within region. Basically, whatever comes in goes out. Whatever comes in has to go out. As is, it cannot create more. It cannot like create more things. Neither can it just disappear inside the system. So, for example, what is an example of destroyed? If a packet is, so for example, if some request has to be dropped, it should be dropped outside the system. We cannot have a drop inside the system where it then goes like this. We cannot have timeouts or something like that by which some requests just die. We can also not have a software system. For example, if this is like some batch processing system where one job creates multiple jobs, some parallel system, parallel processing system where one job creates other jobs, this we cannot have. So, as long as these conditions are satisfied, then the last condition we need is that the scheduling policy should not be dependent on a priori knowledge of service time. So, any kind of scheduling policy is also allowed as long as it is not picking the customers based on some known service time. And the most famous scheduling policy of that type is the shortest job first. As you know that the server actually picks the customer whose service time is the shortest which is in the queue. So, this kind of a service policy, scheduling policy is also not allowed. Now, as long as these conditions are satisfied, the law is as follows. The law says that number in system is equal to throughput multiplied by what is called as Sojourn time. So, Sojourn for a little slur region is defined as a time from which, so this is t equal to 0 and this is t equal to r which is when they leave. So, the time from entering the region to departing the region, we just call it Sojourn time because in different contexts, it may not be called response time, it may have different names. So, we say Sojourn time. But we will apply to our queuing systems where it is actually a response time. So, yeah, so basically number in the system is equal to throughput multiplied by Sojourn time. Let us look at the same thing in a slightly different way and let us look at it now in the context of an open queuing system where I have drawn this kind of boundary around the system. I am just remember in the previous slide I said n is equal to this, I have just rearranged that to say that r is equal to n by throughput. This is also another way of saying Little's law and furthermore, we know that in a lossless system throughput is going to be equal to arrival rate. So, actually it becomes n divided by lambda. So, I am trying to relate this to now what we wanted response time to be intuitively so called we wanted response time to be related to n and the service time. But if you follow this thing, so response time is equal to number in system divided by throughput which means number in system multiplied by inter-departure time. If customers are departing the system at these points, then these times are called the inter-departure times. So, basically there is a departure rate which is the throughput and 1 over departure rate is the inter-departure time. So, it is very, very non-intuitive that actually response time is number in system multiplied by the inter-departure time. It feels a little non-intuitive. It feels further non-intuitive if I should tell you what it is for the lossless systems which is it is number in system. This is of course average and this is steady state average, steady state average number in system. So, response time, average response time is equal to average number in system multiplied by inter arrival time for lossless. So, that is just how it is, it is a little non-intuitive. But I want to finish the story that we started earlier about what if you just take the number in the queue and multiply it by something to do with the service time. Why does not that work or can we make it work? So, let us revisit the direct reasoning which I am calling sort of the direct reasoning for R. So, relating R and N and the service time tau, can that be done? There are two problems there, why that actually did not lead us to this little slog. The main assumption there that like when I was talking about you and your friend, the main assumption that in that reasoning we did was that when your friend arrives, you will see this average number of customers in the system that is the average that you will see. Now, in queuing systems you cannot make that and in general life wherever there is a queue, you cannot make that assumption that the average that you see at the moments when arrivals happen is the same average that a random observer will see that does not hold and I will give you an example for that. So, we need to calculate the average as seen at arrival times. Now, why does I will give you an example of why this averages are not the same. Think of the dd1q meaning with the tau is equal to 5 milliseconds and 1 by lambda which is the inter arrival time as let us say 10 milliseconds. So, I have a server, a single server here with an infinite buffer which has a fixed remember d stood for fixed deterministic that means fixed service time. So, there is a server which has a fixed service time 5 and a fixed inter arrival time of requests coming to the server is 10 milliseconds. So, consider this timeline of the server timeline and this is t equal to 0 and let us assume initially the server was idle. So, the arrival will happen at first arrival will happen after 10 milliseconds. Once the arrival happens the server will be busy servicing this request for 5 milliseconds and the next arrival will happen at 10 plus 10 equal to 20 milliseconds. So, and then again the server will be busy for 5 milliseconds then the next arrival will happen at 30 and so on. Now, if you look at the unconditional the overall average number of customers in the system obviously we have 0 customers of this in the system for half the time and we have 1 for half the time. So, the average number of customers in this system is 0.5. However, what does the arriving request see? The arriving request is always going to see 0 in the system because it is a fixed system you know that arriving request comes every 10 milliseconds and you know that service gets done every 5 milliseconds. So, an arriving request is with probability 1 going to find the queue empty. So, it will see 0 in the system. So, this is 0.5 this is 0 this is a counter example as to why we cannot assume that an arriving request sees an unconditional number of customers in the system. So, in general we cannot assume that that is one reason why the direct reasoning does not work and then as we were trying to do also for non-memoryless distributions it if this holds maybe we can make it work, but we need to calculate the remaining service time. So, that is not also trivial. So, that is why directly trying to calculate an expression for response time in terms of n and service time may not always work. Luckily for MM1 queue for MM1 queue one thing we know that service time is memoryless. So, this is memoryless and as luck would have it the following also holds. What holds is that arrival see unconditional averages. So, we are not going to do the proof for this, but this has been proved that arrivals actually see unconditional averages. So, for MM1 queue using this stochastic processes that I first just recently talked about we can actually from first principles we can derive the average number of customers in the system and it is given by this rho divided by 1 minus rho. Again I am not going to do this proof, but this is just a given to derive this we do not use little slough. We are not using little slough we directly derive if we directly derive we get n is equal to rho divided by 1 minus rho. And we know that the service time is memoryless and the calculation from first principles and stochastic processes without using little slough gives us n is equal to rho divided by 1 minus rho. Now, if you want to use little slough to calculate response time for this remember little slough for loss less is n divided by arrival rate. So, this is rho divided by 1 minus rho divided by this whole thing divided by lambda and remember that rho is equal to lambda tau. So, actually lambda gets cancelled and you get tau here. So, you get tau divided by 1 minus rho. This is basically the response time for an MM1 queue. Now, by direct reasoning if we wanted to use the same earlier kind of reasoning now we can use it actually because we know that arrival C unconditional averages and we know that unconditional average. So, let us try that and we know that memory remaining service time it is also memory less. So, we can just use tau there. So, arriving customer C is n in the system see n meaning n minus 1 in the queue 1 in service and you are going to be the this n plus 1. So, all these n tau service times are in front of you because remember that the remaining service time here is also tau plus your own service time. So, actually you have n plus 1 multiplied by tau overall response time. Now, if you substitute this rho divided by 1 minus rho here for n and plus 1 you get this how rho divided by 1 minus rho plus 1 is equal to rho plus 1 minus rho whole divided by 1 minus rho and this is 1 minus 1 divided by 1 minus rho. So, all of this just becomes 1 divided by 1 minus rho and I am multiplying it by tau. So, I get tau divided by 1 minus rho. So, this is actually really fun that actually direct reasoning does work your intuition was not completely wrong if you if that is what you thought of and what is I also thought of that is not so wrong as long as these two things those two issues of the remaining service time and the arriving time averages in the system that can be figured out it actually does work out and little slaw and direct reasoning actually matches for M M 1 cube. So, there is a lot to digest here little slaw is a very small law, but it has it is many many it is has wide applications and this is what we will do in the next lecture little slaw example and we will also do a case study. Thank you.