 Good morning and warm welcome to the 7th lecture on the subject of digital signal processing and its applications. We will spend a few minutes in recalling what we did in the previous lectures. In the previous lectures, we were looking at what an LSI system is characterized by. So we noted that for a linear shift invariant system, it is adequate for us to know what is called the impulse response or the response of that LSI system to a special chosen input namely the unit impulse. Once we knew the output of an LSI system given a unit impulse input, we know everything about the system. In other words, we know what the output would be for any input. In fact, we proved last time that the output, the input and the impulse response are related by an operation called convolution. The convolution of the input sequence with the impulse response sequence results in the output sequence. Convolution is an operation in its own right. What I mean by that is independent of the fact that it is an important operation in the context of linear shift invariant systems. Convolution can be thought of as an operation between two sequences irrespective of where the two sequences come from. In fact, there are reasons why we may want to think of convolution in that manner. And let us take a couple of situations where we need to deal with convolution as an operation independent of the context of LSI system. In fact, let us take a situation where we have cascaded to LSI system. So let us consider a situation where we have an input X of n being applied to one LSI system. Let us call it S1 with impulse response H1n, the output of which goes to a second LSI system. Let us give it to the name, give it the name S2. And here the impulse response is H2n and this results in the output Yn. This is called a cascade interconnection, a cascade or a series connection of two LSI systems. Now it is obvious that if we use the symbol star to denote convolution, then Y of n is equal to first Xn convolved with H1n and the resultant convolved with H2n. Here we have two convolution operations following one another to relate the input to the output. And there are several questions that we can ask. One question is what would happen if I interchange the order of these systems? So if I were to interchange S2 with S1, would it yield something different? Right? The other question that we need to ask is can I replace this combination, this cascade combination by one single LSI system? Right? So let us put these two questions down. Question one, what would happen if I interchange S1 and S2? And question two, is there one system, let us call it S, which can replace S1 and S2 equivalently. Now we will answer the second question first. In fact, the answer of the second question would emerge from trying to find out whether I can change the location of the bracket. You see? What I mean by that is suppose it were to be true, that means I could replace S1 and S2 in cascade by a single system S perhaps with impulse response Hn. Obviously the output would be Xn convolved with Hn, is not it? And we need to find out this Hn. And what do we actually have Yn to be? Actually Yn is Xn convolved with H1n followed by a convolution with H2n. So my difficulty is the place where the bracket lies. We shall show now that this is equal to Xn convolved with H1n convolved with H2n. In other words, we can change the position of the bracket for convolution. Is that correct? We shall prove this. This is what we shall now prove. And this property is called the property of associativity. So now we are looking at convolution in its own right, associativity of convolution. So in general what we need to show is that Xn convolved with H1n and here we do not necessarily imply Xn, H1n or H2n to be inputs or impulse responses. They are any sequences really. Now let us take the left hand side. I shall indicate to you the modus operandi of the proof and I shall leave some steps of the proof to you to complete. Is that right? So modus operandi of the proof or the method, the process to be followed to prove this is as follows. Consider the left hand side. Now Xn convolved with H1n can of course be written as summation on all k Xk Hn minus H1n minus k. Now essentially this is a function of n. This is not the whole left hand side. I mean this is just a part of it. So I am just, let me, so we are processing the left hand side. That is what I mean by putting this double line here. This is not done. We have just written one part of the left hand side. Now we complete this. So we have Xn convolved with H1n, further convolved with H2n to be summation on k. Now you see here I need to bring in one more variable. So I would not write n now. This is k but here I need to introduce one more variable. Let us call it l. And we put a summation on l now from minus to plus infinity and H2n minus l. Is that correct? This is the expression. This is the complete left hand side. Any doubt so far? Yes? Now we write the right hand side. And the right hand side is of course Xn convolved with H1n convolved with H2. And this is very clearly summation k running from minus to plus infinity Xk. And I need to write, you know, I will use this to denote the sequence obtained by convolving H1 and H2. This sequence evaluated at n minus k. Is that right? Whatever that sequence be. Now when you evaluate that sequence at n minus k, we know how to write the expression for the n minus kth sample of that sequence. We need to use a summation to write that sample as well. The n minus kth sample of the convolution can be obtained as follows. So this is summation k running from minus to plus infinity Xk and then I need to put a bracket. And I need to have let us say an m. An m running from minus to plus infinity H1n and H2n minus k minus m. Isn't it? I am evaluating the convolution at n minus k. Is that right? So what I need to do is to now bring these two expressions together here. I have this expression for the right hand side and I have this expression for the left hand side here. And I need to compare them. Let us write them on the same page with RHS which is summation k running from minus to plus infinity summation m running from minus to plus infinity Xk H1m H2n minus k minus m. And now it is very clear to us how to prove the equivalence of these two expressions. You see what we really need to do is to make a substitution of variable here. If we replace the variable l minus k with the variable m, you know the summation of k is common to both. As far as the summation of l here is concerned for any fixed k, you know remember the summation is from all k equal to minus to plus infinity. That means the summation is on all integers k and all integers l in the left hand side. And all integers k and all integers m on the right hand side. Is that right? And Xk is the term common. Now if we replace m for l minus k, then it is very clear that m minus l, what we need to do is to replace l. So l is then m plus k. And of course now it is very easy to see that n minus l would then become n minus m minus k. But the only thing we need to justify is that the summations are also correct. And that is very easy to see. You see k is as it is. k runs independently from minus to plus infinity. The question is what happens to m? For any fixed k when l runs over all the integers, m also runs over all the integers. So m which is l minus k here runs over all the integers when l runs over all the integers or vice versa. You see for a fixed k, l and m would concurrently run over all the integers. Of course not the same integer. But they would run over all the integers independently. And therefore one can replace this summation, double summation on k and l over all the integers by the double summation on k and m also over all the integers. And therefore the left hand and the right hand sides are equal. And that proves the property of associativity of convolution. Now we need to spend a minute in ensuring that we have no doubts about this proof. Do we have any doubt about the proof? Because it is a very important conclusion that we have drawn. So we have concluded that convolution is associative and that answers our first question. Convolution is associative. This is a more general conclusion we have drawn. And of course its implication for the context of LSI systems is that S indeed exists. The equivalent system exists. And that equivalent system has the impulse response h1 convolved with h2. Very interesting. Now we need to answer the second question. The second question is can I interchange the order? And that amounts to asking. You see now in a way we have also got an interpretation. We know that together the impulse response of the equivalent system is h1 convolved with h2. And if you could interchange the order then the impulse response would be h2 convolved with h1. So in other words we are asking whether convolution is commutative. In other words would h1 convolved with h2 be equal to h2 convolved with h1. Now please note that here I am abusing notation a little bit by suppressing the dependence on the integer index. I am just writing. And anyway you know in a way it is not too much of an abuse because ultimately convolution is an operation between sequences not between numbers. Is not it? It is not too much of an abuse of notation. So let us answer this question. Once again let us look at the left hand side first. So left hand side is essentially summation k running from minus to plus infinity h1 k h2 n minus k. And here again we make a very simple substitution. We substitute n minus k by l. And we note that k running from minus to plus infinity leads to l running from minus to plus infinity for fixed n. This is easy to see. For a fixed n if k runs over all the integers then n minus k which is l also runs over all the integers. And therefore we can make a replacement here. We can replace n minus k by l and that is very easily what we want on the right hand side. So replace n minus k by l which means k is n minus l and of course then we have summation l from minus to plus infinity h2 l h1 n minus l which is essentially the right hand side. And therefore we have concluded that the left hand and the right hand sides are equal and convolution is commutative. Yes, there is a question. Yes. Yes. So I will repeat the question. The question is you see we said that in this case for example when k runs from minus to plus infinity n minus k would invert so to speak. Or in other words it would seem to run from plus infinity to minus infinity. Well that is correct. I mean what I am trying to say is the set of integer the entire set of integers is covered by k and it is also covered by l. Though not necessarily in correspondence that means when k is at a particular integer l is not the same integer that is true. But there is a 1 to 1 correspondence. So for every k there is a unique l for every l there is a unique k and not only that the entire set is spanned. You know when you are dealing with infinite sets you have to be careful. So in fact the question has raised an important point. When you are dealing with infinite sets one must be sure that the infinite sets are equivalent. And in infinite sets you cannot simply you know conclude they are the same unless you can draw a correspondence. That means you must be able to if you claim that two infinite sets are the same are equal then you must be able to draw a 1 to 1 correspondence between those two sets. I must be able to make handshakes between the elements of the two sets and here you can do that. You can make a handshake between elements of the set k and elements of the set l a 1 to 1 handshake. So of course the handshake is not between the same elements but between different elements. Is that clear? Is that clear? Are there any other questions about this proof? So it is a you see we must it is very important that we clarify questions as we go along otherwise it would lead to gaps in understanding and weaknesses in foundation as we proceed. And a weak foundation leads to cracks on the upper stores upper storey the upper floors. Is that right? So foundations must be strong and then that does not allow cracks to seep in on the upper floors of a building. So are there any other questions on the proof so far? Yes is the proof very clear to everybody? So what is our conclusion? The conclusion is that convolution is commutative and in particular what does it mean for LSI systems? The order of a pair of LSI systems in cascade can be interchanged that answers question 1. Now I leave it to you as an exercise to prove the following that we have whenever we have multiple LSI systems in cascade let us say n capital n such systems in cascade any interchange any reordering is permissible. In other words the order does not matter if you have a set of LSI systems in cascade not just two of them any number of them then any reordering of those LSI systems in cascade does not influence the overall input output relationship. And to prove this you shall require both commutativity and associativity of convolution. Is that right? So please use commutativity and associativity to prove this by mathematical induction or any other method that you chose. Is that right? So I leave this to you as an exercise.