 Right, so I said last time that we would look at some physical examples, simple physical examples of birth and death processes. So we will do that today but to set the stage let me remind you of what the basic equations were. So we have a random variable n which labels states for instance and takes on integer values and I said there are 3 possibilities either n runs over all integers or over non-negative integers or over a finite range of integers okay. We will look at these cases separately but this n has a probability p sub n at any time t such that the rate equation for this probability density probability is governed by some gain terms and some loss terms and the gain terms were just for pictorial convenience we put this on a lattice and this was site n or state n and you could move to the right with rate alpha n or to the left with the rate beta sub n the functions of n in general and then the rate equations for this probability were alpha n minus 1 p n minus 1 plus beta n plus 1 p n plus 1 minus alpha n plus beta n times p n those were the rate equations okay. Now of course if you have boundaries then you have special equations at the boundaries for instance suppose you have a case where this thing stops say at the point 0 this site 0 this is 1 etc and this is some state n then of course you do have a rate of jump up there which is alpha naught but there is no corresponding rate here. So this means that beta naught is identically 0 on this site and there is no minus 1 site therefore formally this 2 is not there alpha n minus 1 is 0 also identically similarly out here it is clear that there is a rate beta n going to the left of course there is a rate alpha n minus 1 going to the right but there is no rate alpha n and this is identically 0 and there is no rate coming in so beta n plus 1 is identically 0 which means that at the ends if you have to end point 0 and n you have to write special rate equations and these equations are dp naught t equal to in this case it is clear that sorry alpha minus 1 is identically 0 there is no state minus 1 okay. So this would be if I put n equal to 0 this would be a 0 here but this one is perfectly alright so this is beta 1 p1 minus and since alpha since beta naught is 0 this is alpha naught p0 that is the equation when n equal to 0 on this site similarly at the other end if I put little n equal to capital n you have dp n over dp of t this must be equal to well alpha n minus 1 and that is perfectly alright so it is flowing in from n minus 1 p n minus 1 but then if I put capital n beta of n plus 1 that is identically 0 this term is absent and then correspondingly this term here alpha n is 0 so minus beta n pn these are the special equations at the boundary points okay. So when you write these rate equations down at the end points because of these boundary conditions you have to put these two equations in and then solve the problem okay of course one or the other boundary may be missing may go all the way to infinity we do not care in which case this would be the general equation okay. Now of great interest as I mentioned would be the steady state or stationary solution to this when the left hand sides the time derivative is 0 identically and then you ask what is the stationary distribution out here that is of great interest but before that let me also recall to you something else that we deduced which is that with given general general functions alphas and betas you still can discover that d over dt of the average value n of t this quantity was equal to be found a formula for this and if I recall right it is equal to alpha n minus beta n just check if that is correct this is I believe what we found for the average value similarly d over dt of n squared of t was equal to twice the average value of n times alpha n minus beta n plus alpha n plus beta n those two equations are valid in general for arbitrary alpha n's and beta n's. Now the statement I made was that in the special case in which alpha n is a linear function of n and so is beta n then it is easy to see from here that this right hand side here involves the average of n once again nothing more than that and similarly this thing here involves the average of n and n squared and therefore you have a close set of equations and you can solve them with whatever initial conditions suppose you start in the state n0 n0 for instance you would say this P n is a delta function at n equal to n0 that is your initial condition and then you can solve these equations in that case n of t would be just n0 to start with and you can write down close solutions for these equations even if you cannot find the full distribution itself you can still solve for this incidentally if alphas and betas are linear functions in n it is not very hard to actually solve these equations explicitly okay if they are constants then of course the solution is more or less on the lines of what we had earlier namely we had for n running over all the integers we had the random walk the bias random walk problem in general but there are other variations of it depending on whether you have finite range or infinite range and so on okay. So this much we had earlier now let us look at some specific instances of this whole thing one problem we can do direct right away is to ask what happens if I have a semi infinite range and I look at the random walk problem and to make it interesting for instance you could say this is a charge particle and there is an electric field in one direction which causes it preferentially to jump in one direction than the other or to look at the problem of sedimentation you have a column of fluid and you have particles moving up and down like in the atmosphere here and then the question is how does the density distribute itself the actual physical density of particles as a function of the height under gravity for instance we know that the answer is if you have a column of fluid which is an equilibrium and at constant temperature we know that the density increases exponentially as you come downwards or decreases exponentially as you go upwards this is the famous barometric distribution. So let us see how that comes about here directly you can see that from the random walk problem so if you like it is the problem of sedimentation and in this case let us put the lattice running upwards for instance so here is n equal to 0 and then 1 2 and so on upwards and I would like to know what does this look like now of course if I make this lattice constant go to 0 and take suitable limits I actually get a continuum diffusion model but we will try to see what the random walk model itself says explicitly before that we can do something even more general and that is the following let us assume that you have this n running from 0 to infinity say or 0 to capital N we do not care so it is either a semi infinite or a finite range and let us try to find out what is the equilibrium distribution whether we can actually write this out explicitly or not the question is can I take this set of equations and write out an explicit formula for the stationary distribution so let us do that first stationary and we are going to consider the case n greater than equal to 0 could be infinite could be finite go up to some capital N we do not care for the stationary distribution we need to write put these all these time derivatives equal to 0 that is the stationary distribution. So let us look at this last this equation here this immediately says that P beta 1 P 1 stationary equal to alpha naught P naught stationary because this is 0 so it immediately says P 1 stationary equal to alpha naught over beta 1 P naught stationary and then we go to the next equation and what is that say it says if I set n equal to 1 here it says alpha naught P naught plus beta 2 P 2 minus alpha naught plus alpha 1 plus beta 1 P 1 all of which are supposed to be stationary equal to 0 so we get P 2 stationary with the beta 2 here equal to alpha 1 plus beta 1 P 1 stationary minus alpha naught P naught stationary and for P 1 let us just write alpha naught over beta 1 P naught stationary so that comes out minus 1 P naught stationary and what does that give us the beta 1 cancel so it is alpha 1 over beta 1 there should be an alpha naught I took out an alpha naught so there is an alpha naught P naught stationary therefore it is this alpha 1 alpha naught or beta 1 beta 2 P naught stationary now put that in the third one in the equation for n equal to 2 and so on and so forth and it is easy to see what the pattern is going to be it is going to be exactly what we got from this first this equation for P 2 and we end up with a statement that P n stationary equal to alpha naught alpha 1 dot dot dot up to alpha n minus 1 over beta 1 beta 2 dot dot dot up to beta n P naught and this is true for n greater than equal to 1 so we are done we need to find P naught stationary and what does one do for that normalize normalize the probability so we require that summation n equal to 0 to whatever limit P n stationary should be equal to 1 so this implies that P naught stationary times 1 plus this guy so 1 plus summation n equal to 1 upwards alpha naught alpha 1 dot dot dot up to n minus 1 over beta 1 beta 2 up to beta n equal to 1 and the matter is over so as long as this converges as long as this converges remember that you have do have conditions on these alphas for this to be finite in the case when it goes up to capital N there is no problem this summation ends at capital N but in the case where it goes all the way to plus infinity you need to have this converge this series converge and then you have a finite P naught stationary and therefore a finite value of P n stationary immediately notice that this is valid we have not made any assumption that the alphas are linear function in n or a constant in n we have not done that at all it is still valid okay so we have a lot of information on the stationary distribution if it exists directly and it is some algebraic function of all the rates the alphas to the right and the betas coming to the left okay now let us look at what happens for the case of sedimentation okay what is this sedimentation problem it is a random walk in which under gravity you have a preferred motion to the left so you have these molecules of air they being buffeted around we are only talking about the z coordinate now there is a probability that they get kicked up or down etc but the fact is that the probability of a downward transition is greater because of a constant force of gravity and in this case it is clear since the force of gravity is constant acceleration due to gravity is independent of the height it is immediately clear that the bias towards the bottom downward direction is constant so the beta n is actually independent of n and so is the alpha n it comes from thermal fluctuations in general so the sedimentation problem is modeled by saying that take I take a lattice and let us call this state 0 1 2 site 0 1 2 3 4 etc and in this case alpha n equal to some alpha beta n equal to some beta constant independent of n such that beta is greater than alpha there is a greater probability of jumping downwards than upwards so while this is alpha this is beta and we have taken the boundary into account already in doing this we have said that there is no transition below 0 there is no site down there so it corresponds to what are called reflecting boundary conditions you cannot go you cannot penetrate there is no current that is going from 0 to minus 1 anything like that then what does this solution look like what does this stationary solution look like it says P n now n actually labels the site the height above ground this P n stationary is equal to some P naught stationary but the fact is that this is equal to how we can equal to alpha naught to alpha n minus 1 that is alpha over beta to the power n P naught stationary and that is it what kind of distribution is that it is a geometric distribution there is a constant alpha over beta and beta is greater than alpha and the normalization is trivial here is the normalization this is the summation from n equal to 1 to infinity of alpha over beta to the n so what is the normalization like in this case P naught stationary 1 plus 1 to infinity of alpha over beta to the power n 1 equal to this equal to P naught stationary 1 plus alpha over beta minus alpha which is beta over beta minus alpha so this immediately tells us that P naught station piece n stationary equal to it is a geometric distribution so this is actually telling you that the probability of find finding yourself at a greater height is exponentially decaying as the height increases which is the same as saying that in thermal equilibrium the density is going to be an exponentially decaying function of the height that is exactly what it is because this fellow here can be written as e to the power n log alpha over beta but remember alpha smaller than beta so the log is negative you write this as minus n log beta and that is an exponentially decaying function of the height n labels the height now okay on a regular lattice what would happen if I go to a continuum what would be the parameters involved what would what do you think would be the parameters involved in a physical problem where I have continuum diffusion and we will do this explicitly write the diffusion equation down but what do you think would be the parameters involved these are imagine these molecules to be spheres or something like that dropping under gravity there has to be a characteristic drift velocity due to this gravity what would that be governed and there is a diffusion constant that is what is kicking the molecules up and down so there is certainly a diffusion constant D and what is the physical dimensions of this D what is the physical dimensions of this diffusion constant yeah remember that the obeys a diffusion equation of the form delta p over delta t equal to D del squared p is a diffusion equation of that kind for either the probability of density of finding a particle at some point or for the concentration in the macroscopic picture right whatever it is that p cancels out on both sides so that does not play a role in the physical dimensionality of D so what are the physical dimensions of D length squared over time so this guy here equal to L squared t inverse there also has to be a drift velocity what would that depend on it would depend on the size of the particle on the radius what else would it depend on there is gravity so it will certainly depend on gravity what determines is drift velocity I take a very light part of ball bearings or something like that and put it in oil drops down at terminal velocity what determines the terminal velocity when the viscous drag is balanced by the gravity that is it and what is the viscous drag there is a Stokes formula for the viscous drag right so clearly you have a formula which says 6 pi assuming that these guys are all spheres etc some radius a a times eta that is the viscosity times the velocity right let us call that C the limiting velocity or the drift velocity that is equal to mg so when these two are balanced you have the drift velocity C the terminal velocity so it depends on mg a and eta the viscosity right but there is a quantity of dimensions velocity this is your drift velocity so what do you think the probability density is going to go like we have got thing here in the continuum in the discrete case but now I am going to ask for p of a height z and this is stationary a height above ground so instead of the variable n running from 0 1 2 3 upwards I have a height from the ground levels equal to 0 going upwards what is this going to be proportional to where you can read it off from here this n is going to be replaced by x or z the vertical coordinate and it is got to be a dimensionless quantity whatever is sitting in the exponent so it has to go like e to the power minus something times z got to do that and that something cannot be time dependent and it must be a quantity of dimensions 1 over length and what is the quantity of dimensions C over D right this famous Peclet number or whatever you call it so C over D that is the barometric distribution of density in the atmosphere assuming that the whole thing is at constant temperature but we already see that in this random walk model you already get an exponential decay as you go upwards and there is a boundary at the floor you cannot go below by the way if that is the equilibrium distribution of the height it immediately tells you here is a case where the mean is not equal to where the probability peaks the mean height of the atmosphere the mean height is certainly not equal to the ground because we would all choke otherwise if everything is concentrated at 0 right so you do have an atmosphere all the molecules do not come and sit down there what is that due to thermal fluctuations it is due to fluctuations about the mean about the most probable value or the mean or whatever so it tells you immediately that fluctuations play a huge role very very important otherwise this density is monotonically decreasing with Z as Z increases and the most probable value is Z equal to 0 but everything is not sitting there there is a finite value at which the mean exists what would that depend on what do you think that depends on well it looks like C has a viscosity sitting here but there is another relation called the fluctuation dissipation relation which also depends on the viscosity the diffusion coefficient also depends on the viscosity and the viscosity cancels out there and what does equilibrium thermodynamics tell you it says that if you have an energy level epsilon the relative probability of finding that epsilon is proportional to e to the minus epsilon over k T right what is epsilon for a particle that is at a height Z above ground mg Z so the whole thing goes like e to the minus mg Z over k Boltzmann T right so the characteristic the characteristic length scale in this problem for our atmosphere is k T over mg let us see how big that is to see if this is coming out roughly right or not otherwise the whole calculation is meaningless so k T over mg how big is k T what is how big is k 10 10 to the minus 23 so let us work in those units so 10 to the minus 23 this is 10 to the 2 say T in absolute temperature of the order of 300 Kelvin so 10 to the 2 divided by m what is m mass of a molecule nitrogen molecule for instance how big is that well the mass of an electron is 10 to the minus 30 kilograms mass of a proton is 10 to the minus 27 kilograms and we got in a nitrogen molecule you have whatever be the molecular weight how much is that 28 or something like that so multiply by another 10 so 10 to the minus 26 and then gravity 10 of the order of 10 so this is 10 to the minus 25 this is 10 to the minus 21 10 to the 4 10 to the 4 watt meters 10 kilometers absolutely right bang on that is indeed the more or less the extent of the atmosphere okay so very simple considerations but it tells you that is all it can be with this gravity and this kind of atmosphere these kind of gas and so on and these ambient temperature properties this is all it can be right it is the right ball park so with this is how the barometric distribution arises later we will a little bit later we will solve the diffusion equation with the boundary at the flow on the floor and you will see that is exactly the stationary solution okay let us look at a slightly more complicated exercise and this would be when you have some dependence on n so let us look at chemical reaction in which a species A goes to B and let us suppose B also comes back to A with some rates and I do not know what notation used for rates but the standard one is K and this is K prime so we have molecule A going to B and B coming back to A and the product you are interested in the product B and let us suppose these reactions in extremely simple reaction we will assume there are some enzymes or whatever it is promoting catalyst promoting this reaction and let us assume in the simplest case that you have a situation where you got a huge reservoir of A and you are trying to extract B from it and that the reaction the depletion of A is insignificant so we have such a large reservoir of A that the number A practically remains constant okay then the rate equation for the population of this guy number of molecules is n it is a random variable the function of t and we would like to know what the stationary distribution is what is the average number etc etc in this case you already can compute the average number but we need a model for this whole business so in this case let us assume that each molecule each molecule B there is a certain rate at which it is being formed so this problem alpha n which increases the value of n is proportional to K times that is the reaction rate times the population of A but that is some huge number so let us write it this is equal to K times n A equal to some constant it is not changing significantly on the other hand beta n is K prime and proportional to the number B each of them has a probability or rate K prime of decaying so this is beta some n what happens now we can write down the rate equation for the change probability pn but let us look at what the stationary state looks like in this case so p stationary pn stationary apart from some normalization factor is going to be proportional to alpha 0 alpha 1 etc up to alpha sub n minus 1 that is just equal to K raise to the power n because there are n of these factors divided by beta 1 beta 2 up to beta n that is equal to K prime to the power n n factorial sorry K prime to the n because each of them times n factorial so this whole thing is proportional to K over K prime to the power n 1 over n factorial what kind of distribution is that it is a Poisson distribution so this in the steady state therefore the population of n satisfies a Poisson distribution where the average value depends on these constants the rate constants okay so here is an instance where you can actually get a very powerful result from fairly simple considerations in this fashion let us look at a case where there is dependence on both variables let us look at the example I talked about earlier which is got to do with radiation so the assumption is that you have a quantum harmonic oscillator to 0 1 2 etc etc and you have it in a radiation field interacting with the radiation field with exactly the right frequency so there is this distance is h nu in energy say then the question is what are the rates now the phenomenon is as follows you shine light on it if an oscillator is in this state it can absorb a quantum of light and go up there on the other hand if it is in this state on any excited state it can emit a photon and come down here but there is two kinds of emission there is spontaneous emission and simulated and stimulated emission you have to include both these guys so the model that does it is to say that alpha n minus 1 is proportional to some a times n and when you actually compute the quantum mechanical process by which this absorption takes place you have what is called a dipole approximation a matrix element which gives you the probability for this process to occur and that leads to a rate of jump which goes like this a transition rate which is proportional to n alpha n minus 1 similarly it turns out that beta n equal to some b n and now we ask what is the stationary distribution going to be like in this case so what is p stationary going to look like p n stationary is proportional to or equal to in this case it is alpha naught alpha 1 etc etc all the way up to n minus 1 alpha n minus 1 so it is a times a to the power n 1 into 2 up to n divided by beta 1 beta 2 up to n so this is also equal to b to the n 1 into 2 n is a over b proportion now if the oscillators in the state n okay its energy is n times h nu and what is the relative probability in thermal equilibrium and the system is in thermal equilibrium the radiation field as well as the oscillator what is p n proportional to then the stationary distribution proportional to this fellow has to be proportional to e to the minus beta h nu n h nu over k Boltzmann t that is the canonical ensemble right so a over b has to be proportional to e has to be e to the minus n h nu over k t this is the starting point of Einstein's derivation of Planck's law so you put in all the other factors etc and you end up with the Planck's distribution but notice in this case it was not constants and yet because of this special feature here these factors cancelled out and you still got this geometric distribution we have already seen that the number distribution in thermal light is indeed a geometric distribution we already saw that using both statistics etc I wrote it down explicitly this car is essentially the same thing now let us look at a case where you have a genuine birth or death problem so let us look at so we looked at this radiation let us look at population the simplest population model and a genuine birth or death problem so again n starts from 0 and goes upwards okay the rate equation is something we wrote down already but we need to make a model for the growth of this population so let us say that alpha n this is going to be the rate at which n increases to n plus 1 so let us suppose that each make the simplest model that is all births are independent events of each other and we make a model which is completely trivial in the sense there is a 0th order if you like to say that each individual has a certain rate in which the individual can give rise to one more progeny okay some constant rate so alpha n is obviously proportional to n where is some constant okay and similarly each individual has a constant rate at which the person dies the probability of dying the rate transition probability so beta n equal to b times n that is the total death rate and this is the total birth rate here okay and using this we can now write down what the stationary distribution will look like etc if it exists but let us look at what the average does remember that the equation for the average was d over dt n of t equal to alpha n minus beta n so this is equal to a minus b n of t what does that tell you it says this is going to be exponential growth of population if the birth rate is greater than the death rate this was the famous Malthusian prediction of exploring population okay in this simplest model of models exponentially fast one could of course hope that the variation that the variance will do something will help us a little bit let us try and see what happens to the variance in this case so let us look at the equation for d over dt n squared this was equal to twice the expectation value of n times alpha n minus beta n plus alpha n plus beta n and what does that give us twice and again this is a minus b so the prediction here is exponential if a is greater than b so let us see what this tells us oh incidentally keeping track of the average value of the population only tells you the difference between the death rate and the birth rate or the birth rate and the death rate does not tell you individually what is the death rate and the birth rate that is important okay. So this is a minus b and then expectation n squared plus a plus b expectation n so let us compute what the variance does d over dt let us put n squared of t minus n of t square equal to this is the variance of n let us give it some symbol the sigma square the standard symbol okay so d over dt sigma squared equal to this fellow 2 into a minus b n squared plus a plus b n d n squared over dt and then we got to subtract the time derivative of this so that is equal to minus twice n times d n over dt but d n over dt is a minus b n of t and itself and this factor comes out so this is equal to twice a minus b n squared minus n average squared that sigma squared once again so sigma squared plus a plus b that 2 is increasing exponentially because this guy already is if a is bigger than b and this is a positive term and this guy is sitting on the right hand side so this 2 will explode so this is not helping us any this simplest of models it is clear there is a huge scatter but at the same time the whole thing is the average is growing exponentially fast. Now you can put in all kinds of various mitigating factors and improve this model and so on and so forth but the basic problem of exponential growth happens as soon as the birth rate is increased greater than the death rate what is the reason okay one can look at other models which have competition which have different competing species etc etc but the fact is that in the most elementary k instance this is what is going to happen here it is worth noting that if you want to measure the birth rate and death rate separately you need to measure both the variance as well as the average number because this is going to depend on this guy so you can get information on both a and b separately once you do this okay there are a lot of other models there are a huge number of models which you can write down in general as I said if the system is linear in n then the problem is actually solvable explicitly and in the we have restricted ourselves to these one step processes in which case this transition matrix is a tridiagonal matrix and there are lots of special tricks to take take care of this system the tridiagonal matrix for one thing it is not symmetric to start with but you can actually make it symmetric by a simple trick and then you can go some distance in writing down approximate solutions etc but again I want to emphasize that when it is linear the problem can actually be explicitly solved although the coefficients are not constants depend on n you can still solve these matrices okay now there are further delicate questions as to the kind of boundary conditions you put the nature of the boundary conditions they are absorbing or reflecting and so on we will come back to this but not in the context of birth or death model but in the context of continuum model of diffusion we write the diffusion equation and look at it in the context in that context we look at absorbing boundaries reflecting boundaries and so on okay so when you have extinction for example you would have what is called an absorbing boundary and that leads to its own interesting features so we will look at that by and by very shortly when we talk about continuum things so I am not going to go further with these discrete models random box but we will now talk about continuous Markov processes where we are going to define once again probability densities and conditional probability densities and use some physical examples to illustrate what happens in such cases so we start off by saying that the processes we are going to look at continuous processes and as usual I will denote by x the random variable and by x curly x the set of values in the sample space assumed without for the most general case to run from minus infinity to infinity we look a little later at cases where this x is a vector for instance position and velocity or position and momentum etc but for the moment it is just a scalar variable here then exactly as we said in the case of Markov processes in which the variable was discrete we are going to define a probability density functions of this joint probability density functions and in the case of a Markov process the most important one is going to be the probability density that you have x at time t given that you had some x0 at time 0 and if you look at a stationary at some time t0 for example if this is stationary then this becomes p of x t minus t0 x conditional density and we look at the one time probability which is this should really have been a p 2 but I am going to omit this it will become clear from the context which one I am talking about and then you have a p1 of x, t and of course if it is stationary it just goes to a p of x and exactly as in the case of the discrete variable I said if there is sufficient degree of mixing in the system then this tends when t minus t0 tends to infinity this conditional density tends to this density here depends only on x does not depend on the time argument okay. As before we write a chain equation down the chain equation in this case is once again p of x t x0 equal to a summation which becomes an integration now overall intermediate values dx prime p of x t minus t prime from x prime p of x prime t prime x0 where on the time axis here is 0 here is t prime and here is any t prime in between this is the Chapman Kolmogorov equation or chain equation as before we try to convert this into a master equation which is an integral differential equation by defining a transition probability w of x from a any value x prime to a value x and then what does this equation go to this leads to this master equation delta p over delta t of x t x0 this is equal to an integral over dx prime and then inside here you have a w of x x prime p of x prime t x0 minus the loss term which in this case is x prime x this is our master equation and the task is to try and solve this equation we have assumed stationarity otherwise this equation will have t dependence is here in these transition rates even if it is Markov we have assumed the whole thing to be stationary and if you specify these equate these quantities now you got to specify functions here then in principle I have an integral differential equation which however is linear in this unknown quantity in the conditional density and the idea is to solve this. So we will take it from this point we want to start here and see what are the possible ways in which we can solve this in various cases and then we look at some physical examples let me stop here.