 So, this is lecture 23, so we saw quite a few things in the previous lecture, a lot of interesting new things which are quite crucial in understanding why LDPC codes work in practice. Towards the very end, I showed you a plot, so maybe I will insert that here, it is a picture, so this picture looks quite big, so then at the end of the day after everything has been said and done with all these analysis and thresholds and all that, finally this picture is very crucial, so you should be convinced about the performance of LDPC codes immediately after seeing this picture, so you can go back and work with a lot of these things like thresholds and all that and finally that is very important that you know all that, but at the end of the day this is the picture which justifies LDPC codes, so for instance we have this wonderful threshold analysis which is based on this IID assumption which is based on the tree like neighborhood or no cycle free neighborhood up to depth L, so here I have codes of block length 30, 100,000 and 10,000 and I have constructed them, I have constructed so that there are no length 4 cycles, so I have taken care to remove length 4 cycles and each and every one of these parity check matrices has a lot of length 6 cycles, they have quite a few, let me not say lot, they have quite a few length 6 cycles, so what does it mean, moment I say there are length 6 cycles, length 8 cycles, up to what iteration do you have IID assumption being valid, only up to 2 iterations, so third iteration onwards you will start having problems with your IID assumption, but still what happens here, what happens here in spite of the fact that you have violations of the IID assumption at the beginning with the third iteration, I am doing 10 iterations here and the performance seems to track the threshold very closely, so that is pretty much the state of knowledge today as far as analysis of LDPC codes, we know that threshold analysis is valid only under the IID assumption which requires cycle free graphs, if I have 10, if I am doing 10 iterations for the analysis to be accurate, minimum length cycle should be 22 or so, otherwise I cannot talk about threshold being a valid thing, but what am I seeing in practice, I am seeing in practice that even if there are several length 6 cycles, I can do iterations up to 10 and threshold is still meaningful, so if you come up with a theory to extend the threshold analysis with cycles you will become very, very popular, so it is considered a very non-trivial thing to do, but you can always draw mean do simulation plots like this and justify why this is very useful, so this plot the x axis is transition probability, so as you move to the left the transition probability becomes better, so to the right of the threshold you have very poor BR performance and to the left of the threshold it falls down, not really, oh you have to see if it is worse than the input transition probability, not necessarily the threshold, it could be, so what is the moral of the story, at the end of the day threshold analysis is very useful even in very practical scenarios, even when you cannot guarantee that the IID assumption holds, threshold analysis is still useful, maybe one day somebody will show why it is useful, but for now we can accept that it is useful even when the IID assumption breaks down, so that is the first point and the next thing is I thought I will be able to show you a nice animation, I thought I had saved it in e if I want to, it looks like this folder does not have it or maybe it has, let me see, okay it is there, so I am going to show you an animation, hopefully it will work, okay, so I have to explain quite a few things, can you see it large enough, okay maybe I will blow it up a little bit, I am sorry, okay but that is good enough, you can see what the animation is, okay, so this is, okay I have to explain quite a few things before I launch into this, so far we have been discussing the decoder from a graph point of view, okay, I have always described the messages as flowing from bit nodes to check node or check node to bit node on the edge, okay what does an edge correspond to in the matrix, the ones, okay, so I can imagine at any iteration in step A, okay, there are a bunch of messages that are flowing along the edges, I can imagine that as a matrix, okay, matrix, a sparse matrix with the messages being put on the non-zero positions of my original parity check matrix, okay, maybe this is a little difficult for you to imagine but let me start with very simple things first, okay, so this is the 30 comma 15 parity check matrix that I have put up here, so what do I mean by saying this is the matrix, okay, so remember, yeah, okay, so I guess it is clear to you, okay, so it is got 30 columns and 15 rows and what do the, what are the green blocks, the green block represents a 1, okay, black means it is 0, okay, so in the graph the green blocks represent edges, okay, so in step A of iteration 1, I will put values on these green blocks, what values will I put on these green blocks, the actual messages that are flowing on the corresponding edge, okay, for instance, from the channel what do you receive, you receive 30 bits, okay, each of those bits I can associate with each column, what will happen in step A of iteration 1, all the messages will flow on that column, okay, so you will see I have assumed all 0 codeword, okay, and my probability of error is quite low, so you will have very few errors in each out of the channel, okay, and what I am going to do here is, so you can read, if you can read the title, it says block 1 output after iteration 2, okay, so I am going to backtrack a little bit, okay, so this is not going to allow me to backtrack, how do I backtrack with this? Okay, so clearly you guys are better at Windows media player than I am, okay, so then what do I do, it's not changing, I have to play, is it, but I want to stop immediately at the first frame, okay, but it's going, oh my god, it's gone, third button from the left, no, it won't play, how do I play it slowly, is it possible? Okay, anyway, so let me try to be very quick with my play and pause, okay, okay, but somehow it's gone to iteration 1, okay, I want to, okay, so I guess it's not too bad, but this is not the thing I want, there is one more frame before this, okay, so I think the quick time player is better, do we have quick time player on this? No, it's a problem with these Windows things, there should be something, no, that will play it frame by frame or something, play speed, okay, so maybe this is not a good, okay, so this is the plot I wanted, okay, so I've called it iteration 0 block 2, so when I say block 2, it's a fresh block that comes out of the channel, so each block is 30 bits, right, my code word length is 30 bits, so I took 30 bits out of the channel, okay, what happens in step A of iteration 1, okay, I've called it iteration 0, what happens, what are the messages that are flowing from the bit nodes to the check nodes, okay, so you see from here you can easily figure out how many bits were in error out of the channel, two bits were in error, right, so what is red now, red is 1, green is 0, okay, so this is what, is it clear, okay, reasonably, no, you can't see what's going on, okay, I'm sorry, okay, so now, okay, what is this matrix now, okay, so initially I showed you the matrix and I said greens were 1s, okay, that's the parity check matrix, now I want to use the same matrix to represent the messages that are flowing on the edges, okay, is it clear, I want to use the same matrix to represent the messages that are flowing on the edges, okay, so this is how we implemented in practice, right, when you actually implement, write a program to implement message passing decoder, you have a sparse matrix with non-zero values on the non-zero locations of your original parity check matrix because that's where the messages are flowing, right, all my rows are my checks, my columns are my bits and the messages that are flowing I represent at the, on the edges, edges are what, the non-zero locations of my parity check matrix, okay, so now don't think of this block as a parity check matrix, this actually represents the message that is flowing from bit node to check node in step A of iteration 1, okay, so iteration 0 is what I called it, okay, let me slowly come to it, so now when I go to messages, any message which is 0, I am going to put as a green block, any message that is 1, I am going to put as a red block, okay, what a black thing, yeah, those are 0 positions in the parity check, those don't correspond to edges for me, so those are not messages, okay, is that clear, okay, so this is how you actually represent the message flow in your program, okay, it is a sparse matrix, it's an easy way of thinking about it, okay, or you can write a graph or whatever I mean, something, this is basically, this is how I have represented the graph, okay, okay, I am only going to show messages from bit node to check node, I am not going to show messages from check node to bit node, okay, just to simplify the animation, I could have added that also, it's not a big deal, but that, I have just shown messages from bit nodes to check node after each iteration, okay, hopefully I can stop it iteration after iteration, we will see, okay, so you see this is what happened after iteration 1, okay, so after iteration 1, the messages that go from bit node to check node are all 0, what does it mean, I have decoded correctly, okay, so you can, let's see if I can go back, I can show you how it worked also, okay, so this is bad, okay, this is block 1, right, okay, it's terrible, I don't know, I am not even able to grab that thing, so okay, anyway, so let's see, okay, anyway, so you get the picture, right, so let me see, maybe the next block, I think we should really work on a better media player than this stupid Windows media player, I don't know, you should get something, yeah, something I can run frame by frame, I don't know how it can't let me run frame by frame, I don't know, it's just ridiculous that it doesn't allow me to run frame by frame, okay, anyway, so I think this is not ideal for this demo, but I'll just run it and hopefully you'll get something out of this, okay, so you see this is, next you'll see block 3, well there were no errors, block 4 had one error which went away, block 5 had one error which is going away, block 6 has several errors, so you see it won't go away, okay, as the iteration numbers are increasing, those errors are continuing to be in the messages, so after a while I say this is my output I give up, okay, block 7, similarly, there are several errors and they keep on spreading, okay, so you see as the iteration progresses, different errors show up in different places and it's very random, it's tough to predict, okay, if there are one or two errors, in many cases it works, 10 iterations I'm stopping, or if I get a valid code work, H times, H times, you take your parity check matrix, multiply with C cap transpose, no, you are doing it, actually, you can, you can see, you'll see at the check nodes you're always evaluating the checks, so you can use that to simplify your, okay, so you can see how it works, okay, and it's, so you can imagine this is for the 30-15, I can also, I could have also done this animation for the 151, it would have been much, much bigger, the boxes would have been much smaller, maybe you can't see it, okay, so I'll try to upload this animation also on the website, maybe then if you have a better media player, you can, you can play it frame by frame and see how it works, okay, so but roughly the idea is clear, right, so this is how, this is how the, the messages would actually look, okay, so it's difficult to do it in class, take a 30 by 15 matrix and fully do it, run it for you, stuff, okay, for each Solomon decoding and all, we can do it, but for these kind of decoders, it's difficult to do, okay, you want the code, let me, I have some ideas about that we'll see, okay, so how do I get back, okay, this is not what I want, this is terrible, I want that bottom thing to come, I don't know how to make it up here, okay, anyway, this is too complicated for me, no, simple, people don't like simple programs anymore, really complicated programs, anyway, so, okay, so, so what am I trying to say, so, so for a very large block length, okay, when you're actually even running a simple algorithm like Gallagher A, all kinds of things will happen and things like what you observed and people are making remarks yesterday that you may never converse, right, you keep on going on and on, you saw that happen quite a bit, okay, some very strange things can also be observed, there'll be only very few instances when you will converge to the wrong code word, in fact, I've done simulations for so long with LDPC codes, I've rarely ever seen unless I designed, I picked a real bad code from the ensemble, I'll hardly ever see convergence to a wrong code word, it'll either converge to the right code word or it'll keep on doing these oscillations back and forth, back and forth, it'll never converge, okay, so those are all interesting properties for this decoder and you can't really analyze them in a very easy way, so it works, right, it's quite strange to analyze, yeah, so, yeah, exactly, so you'll not have problems of wrong convergence with these decoders, well, in practice you won't have, can I prove it, maybe I can't prove it, okay, but in practice, it's very unlikely that you will have it, it's tough to calculate any other probability, okay, all right, yeah, by simulations you can check, okay, so, okay, so the next thing I want to do is go back to this PL and QL things and just speak a little bit more about it, I think I kind of rushed the whole threshold argument towards the end, but hopefully it's clear, okay, so PL was what, we said PL is probability that a message from bit to check node, no, bit to check equals 1, okay, this was the probability, probability of erroneous message from bit node to check node and we had a recursion for this, we came up with a recursion, okay, some recursion which I'll call l minus 1 and this recursion is parameterized by WR and WC, right, I'll put down WR and WC here, just to be very specific, okay, there were terms inside the recursion which involved WR and WC, we in fact did it in two steps, we went from PL to QL and then QL to PL plus 1, okay, so that's how we did that and P0 is the same as the transition probability P, okay, so in fact you can even say this recursion is actually a function of PL minus 1 and P, okay, the reason is that each step P is also involved, so you'll see, right, from when you went from QL to PL plus 1, P was also involved, so if I have to write that properly, this is a very nice and concise way of writing down that recursion, okay, so now when you study this function, you have to study it as a function of two variables in PL minus 1 and P, okay, and there are a lot of interesting analytical properties for this function, okay, and what are its fixed points, what are its, is it monotonic with respect to one argument, is it monotonic with respect to the other argument, and all these properties are very interesting and there's lots of studies based on those things, okay, so I don't want to talk too much about it, I'll simply state some analytical properties of this function, okay, I won't even state it, I'll just say some properties are important, okay, analytical properties such as monotonicity, monotonicity with respect to P and PL minus 1, okay, both of those are interesting, and then fixed points, what is fixed point now, yeah, so fixed point would be PL will be equal to PL minus 1, okay, if you reach a point where the iteration basically converges, right, it doesn't go beyond that, okay, so yeah, the derivative being equal to 0 is also another way of looking at it, okay, so fixed points, etc. can be studied, okay, and there's lots of literature, you'll see, there'll be lots of literature on these kind of things, okay, no, I mean for the function fixed point is PL equals PL minus 1, not for the code, I'm just saying fixed point for this iteration, I'm just studying analytical properties of this function, okay, so you want to derive it, you want to see if it's continuous, you want to see things like that, analytical properties, okay, so fixed points are, there is a connection between fixed points and thresholds which is again very interesting, okay, so then how do you define thresholds now, so okay, so the way I said that is for, depending on P, okay, so clearly if P equals 0, okay, what is PL? PL converges to 0, and if P equals 1, what happens to PL, okay, it'll also be 1, right, so those two you can check, you can put it in that formula, you can plug it into the formula, you'll see if P is 0, it'll be 0, P is 1, it'll always be 1, okay, so from there you can argue, since it's a monotonically decreasing sequence bounded away from 0, there will be a minimum value of P, minimum or maximum, yeah, maximum value of P for which PL will still converge to 0, okay, and that's how you define the threshold, okay, so let me write down how the threshold is defined, okay, P star is maximum P such that PL converges to 0, okay, so in fact people don't use the maximum here, the reason is, yeah, use supremum, you don't want the maximum to be in this set, so if you do that then you can't define it properly, you can always keep on going over and over, so if you say supremum, that's the right thing to say, okay, so anyway, but if you're not worried about it, just forget about it, just say maximum, that's good enough, okay, so you start with P equals 0, keep on increasing P, okay, run this recursion, see at what point you're getting the PL to converge to 0, till what point you're getting PL converging to 0, at one point you'll see sharply PL will converge to something non-zero, okay, it will converge to something which is not 0, at that point you declare that as your threshold, okay, the smallest value at which that happens becomes your threshold, this is a proper way of defining thresholds, one can do it, see this recursion is not very difficult, you can implement it in MATLAB and two lines, right, just a simple closed form expression, can implement in a two line and write a simple program to run this recursion, quickly find thresholds for WR and WC, okay, like I said for 3,6 the threshold is 0.04, okay, for other things also you can do it, okay, so that's about the threshold, any questions on how to compute this is clear, right, okay, at least for the Gallagher A algorithm this is clear, okay, yeah, one can study that also, so you can write the whole thing in terms of QL, so instead of studying this recursion I can write for instance, so he's saying instead of studying PL tending to 0 why don't you study QL tending to 0, both of them will happen together actually, this is difficult to have one happening and the other not happening because PL has gone to 0, QL will become 0, if QL is 0 then PL plus 1 will become 0, right, the way we defined our iterations, okay, but, but let me let me reiterate once again, the QL iteration might be useful in some case, okay, because the form of the QL iteration is different from the form of the PL iteration, you write the whole iteration for PL in one way, you'll see that whole expression is different from the expression for the QL iteration, okay, in some cases that might be easier to optimize or study or anything, so for that reason you might study the QL iteration, okay, so that's also equally important, you can write QL instead of PL also for this, okay, so there's no sanctity for the bit to check message being one, you can even look at the other one, okay, any other questions on this? Okay, so the next crucial thing to understand is to understand this analysis in terms of the neighborhood, okay, so that's one more piece of understanding which will give you more clarity about what's actually happening, okay, I've been talking about this neighborhood and I've been associating this IID assumption to lack of repetition in the neighborhood or lack of cycles up to suitable depth in the original graph, okay, so let's try to write down the neighborhood and associate this PL and QL to different stages of the neighborhood, so you'll see there'll be a natural association, okay, so that's very important, okay, so let's see that, okay, so let me write down the neighborhood just the, I'll call it the neighborhood view of Gallagher A, okay, so you have the bit node, okay, so this is one edge which is going from the bit node to some check node, okay, then there'll be other edges, how many edges will there be? W, C minus 1 of them, okay, edges going to check nodes, from each check node you'll have WR minus 1 edges going to bit nodes, right, okay, so these guys will be WR minus 1, okay, so likewise the neighborhood will keep on going, okay, so this is depth 1, no, is depth 1 or depth 2, depth 2, okay, so then you go to depth 4, depth 3, you'll get C minus 1, okay, I know I'm drawing a very bad picture but maybe you can improve this, each of these things will be W, C minus 1 again and then you have, okay, so the depth 4 neighborhood, so likewise you can keep on going and finally you'll have the depth 2L neighborhoods ending up like this and even here you'll have WR minus 1, okay, so this is let's say depth 2L, okay, okay, so how am I going to think of Gallagher A decoding on this neighborhood? So for each bit, okay, this is quite important, there's lots of meaning in this, okay, so for each bit, for the bit on the top, okay, right, for the bit on the top, if I form this depth 2L neighborhood, okay, and forget about the rest of the graph, okay, can I say what will the message be from the bit node to that check node, okay, so if I'm only interested in this message after L iterations, okay, I don't care about any other message, okay, so suppose I isolate my view at one bit node, okay, and say the message that goes from that bit node on a particular edge, okay, I've come to that point, okay, I'm only interested in what happens to that at the L iteration, is it enough to just have this neighborhood or do I need anything else, okay, do you see why that is enough, okay, right, now I have a bunch of bit nodes here, in each of these bit nodes, okay, the last bit nodes at the very end, okay, those are called the leaf nodes of a tree, okay, the leaf nodes of this tree, okay, if I fill out all the received values, okay, and then if I go through and fill out the received values in each and every bit node from there, I can completely figure out all the messages that are going to flow that will affect this final message that I have here, okay, are we able to visualize this a little bit difficult to visualize, okay, so, but this is reasonably crucial, okay, is there any confusion here or is there any, sorry, okay, so, so, so, so, so, so, okay, so how do I, okay, so, so the main objective is Gallagher A decoding, the way we described it is, how did we describe it, we said, okay, bit to check messages are being passed, inject to bit messages are being passed, all of it is happening in parallel for all the bit nodes and all the check nodes, okay, then we wanted to calculate the probability that one message from a bit node to a check node, iteration L was in error, okay, that is what we wanted to calculate, I did that very quietly by saying some IAD assumption, etc., and all that and everybody was happy, okay, I am not saying that is a bad thing, okay, it is good that you believe me and you are happy, okay, but if I have to properly do it, if I have to actually find the probability that a message from a bit node to check node at the L iteration is in error, I have to actually backtrack from that edge, go back to the previous iteration, the previous iteration, previous iteration, figure out all the bit nodes that would have actually contributed to that message and then go through that and calculate it carefully, okay, that is the proper way of doing it, right, so that calculation will come only if you understand Gallagher's decoding in terms of this neighborhood, okay, so maybe this whole depth 2 L picture is too advanced to start with, okay, maybe we start with depth 2 picture, okay, just for the first iteration, okay, so suppose I say the bit to check node message in iteration 1, iteration 1 step A, what are the only bits that are involved in this message? Just that bit itself, right, because this message itself is RI, okay, now if I have to find the same message on the same edge after iteration, in iteration 2 step A, what should I do? I have to look at the neighborhood of depth 2, you see that, okay, because what would have happened, okay, so this would have been connected to, let me fix WR equals to 3, okay, this would have been connected to 2 other check nodes and each of those check nodes would have been connected to 5 other bit nodes, okay, so I am drawing the neighborhood in slightly different way, okay, so if I know all the messages here, all the received values here, okay, and the received value here for all the bit nodes, okay, then I can figure out what this message will be at iteration 2, okay, you are not able to see this, okay, so if I say RI here, okay, RI is the received value here for this bit, what is the message at iteration 2? I cannot say it is RI, obviously it is not RI, I do not know RI, I need to know these messages and these messages, how will I find this message? Okay, I need to know what was passed before, okay, so you have to go all the way up to a depth 2 neighborhood to figure out what message would have come here, for instance, if I say, let us say, let us say all these guys were 0, okay, all the received values corresponding to all these bits were 0, then what would have happened, what would have this message been? 0 again and then I can say this will also be 0 according to Gallagher, okay, with check node, so with check node, this check node you mean, why am I not worried about this check node, because the way I design my Gallagher A, I will never send back anything I receive from that check node, okay, so that is crucial, so I do not have to worry about what happens beyond that check node, because whatever came from there, I am not going to use it when I send information back, okay, so to figure out the message that is being sent in iteration 2 step A, I need to look at a depth 2 neighborhood of this bit node, okay, okay, now I can repeat the same thing, suppose I say iteration 3 step A, what do I need, I have to again figure out what messages would have come before this, so I have to put one more check node here, one more layer here, okay, right, so it is the same, this picture I have generalized here, okay, in the previous picture that is what I have written down, okay, suppose I want to figure out the message that goes from a particular bit node on a particular edge, I have to look at its depth L neighborhood, okay, like you said I do not have to worry about what happens to the neighborhood of the check node here, this side I do not have to worry about, on top of that I do not have to worry about, only to the other side I have to look at, okay, hopefully it is reasonably clear, it requires some careful thought, okay, it is not very simple in the first place, okay, because I have been, I have described the Gallagher algorithm, it is most easy to describe and analyze in this parallel fashion, you may isolate it to one bit node and see what all the other bits involved, it is a little bit difficult to imagine that, okay, but it is important, okay, it is very important. Now I want you to calculate something, suppose I want to calculate the total number of bits that will be involved in the calculation of this message, can you calculate that? All the bit nodes in this neighborhood, okay, the total number of bit nodes in this neighborhood will be the neighbor, the bits that were involved in the message from the bit node to the signal, okay, okay, yeah, only to well know all these other bits will also be involved, no, no, no, no, no, it also depends on what was received there, see if these two match, then see that can actually be RI, right, this message can be RI, if these two do not match, so obviously there is a crucial dependence on every bit node in the neighborhood, not only the leaf nodes, see you start at the leaf nodes, yeah, in iteration 1 you start at the leaf nodes, but then when you go into one of the intermediate bit nodes, the value of that bit node could be the message that is sent, because if these messages do not agree, then you would send only the value of the bit node, right, what is this message? You go back and think about Gallagher A, how did we decide this message? It is equal to RI if the messages do not agree, okay, so ultimately when I do this calculation, the total number of bit nodes that are involved in the computation of the message in the L-th iteration from a bit node to a particular check node is a function of what and what, it depends only on what and what, WR and WC, it does not depend on N, okay, okay, so all these things are crucial arguments in the proofs and all that, okay, the number of bit nodes depend only on WR and WC and L, of course, because L, but it does not depend on N, so I can tend N to a larger and larger number, but what will be the same? The number of bit nodes that are involved in the computational message will be the same, because it depends only on WR, WC and L, okay, these are, this is a fact that is used several times in many of the proofs of LDPC codes, convergence, etc., etc., okay, yeah, even for that you use properties like that, okay, and this WR and WC are presumably small numbers compared to N and you can imagine this being very large, okay, that's the first thing that I want you to think about once more, convince yourself that you understand this relationship between the neighborhood and Gallagher decoding algorithm, okay, next thing I want you to do is I want you to put down P0, P1, P0, Q0, P1, Q1, etc., on this neighborhood plot, what is the, where is the probability P0 going to show up? What is the probability P0 is? Well, which one is an error? The Rs at the leaf nodes, okay, what about Q0? The one immediately above that, do you see that? Okay, so if you have edges here, okay, so this will, this is where P0 will show up, it's the probability that the message from the channel is an error, okay, these messages, any of these messages are an error, the probability that any of these messages are an error is Q0, what about Q1? The next stage, P1, next stage, Q1, next stage, next stage, okay, do you see that? Okay, so likewise as you keep going, you will get this to be PL, this will be QL, L-1, etc., okay, this will be PL-1, etc., do you see that? Okay, and all our calculations, these IID assumptions, everything are valid, if none of these bit nodes repeat, do you see that also? Okay, if I make an all-zero assumption and there is no repetition among any of these bit nodes up to depth 12, then I can assume IID, right, because there is no dependence, they are all identically distributed, okay, so this is how you tie up this whole thing in a solid way to the neighborhood, okay, the neighborhood plays a crucial role in deciding what the message is from at a particular iteration, every iteration, okay, and only the neighborhood plays a role, the other things don't play a role, okay, whatever is outside of this neighborhood, those bits do not affect my message at a traditional, okay, only these bits affect, okay, so it's clearly suboptimal, okay, if you stop at a finite number of iterations, a iteration number is very small compared to the block length, there is every chance that your decoding will be suboptimal, okay, you may not even use all the information that's out there, or it doesn't matter to you if it's good enough, okay, so it's important to even associate this calculation, this analysis P0, Q0 to this neighborhood, okay, what is P0? The probability that the leaf nodes, the message from the leaf nodes are in error, what's Q0? Probability that from the check node to the next level is in error, bit note to the next level is in error, that's what we calculate, okay, I did that very quickly and quietly on the simple description, okay, nobody really asked me any deep questions at that point, which is good, you should understand that first, then go and dig in here and understand this further, okay, the reason why this understanding is very crucial is, eventually we will go to a situation, okay, the next thing we'll do is, we'll move to a situation where we don't expect the graph to be regular, okay, so we will make it irregular, okay, what do I mean by irregular? Some nodes will have different degree, some other nodes will have different degree, okay, then what will happen, okay, this neighborhood will change depending on which bit you look at, right, if you have a regular code, then the neighborhood remains the same irrespective of which bit you look at, right, looks the same, bits will be different, but it will, the structure will look the same, the calculation for P0, Q0 will hold for every bit in the exact same way, okay, right, remember P0, Q0, what was involved in the calculation? WR and WC, why is all that valid? Because this thing looks exactly the same for each bit, okay, when I go to irregular structures for my tanner graph, the first confusion will be, the neighborhoods are not the same, okay, so then how do you do this analysis, okay, so all those questions will come up, at that point you have to do some further approximations and concentrations and all that, okay, so we will do all that, to visualize all that very clearly and see exactly what is happening, you need to keep this picture in mind, you should also keep the previous picture in mind, just do some IID assumption and then do the computation in a very simple way, okay, so we will have to do both of those, okay, so, all right, is this clear, any other question lingering doubts about this neighborhood and how it plays in, it is okay, okay, the next thing I want to comment upon is the all-zero assumption, okay, so why is the all-zero assumption true, okay, what am I actually doing when I do the all-zero code word assumption, okay, so what I need to actually compute is probability that C cap is not equal to C, okay, this is what I am interested in computing, this is my probability of error, what I could do is sum over all codes in C, all code words in C, my transmitted code word, probability that C cap is not equal to U given C equals U times probability that C equals U, okay, I am going to condition it on each individual code word, find the probability that you have an error given that that code word has transmitted, then sum it over all those cases, okay, I am going to assume all code words are equally likely a priority, so what will be this value, 1 by 2 power k, okay, so that is clear, okay, but what about these guys, why should they be all equal, okay, suppose I say no, let me rephrase, let me slow down a little bit, suppose I say this expression is independent of U, it is equal for all U, okay, then what happens, I can even pull that out and say this whole expression will simply become probability that C cap is not equal to U given C equals U for any U, for any 1 U, I compute this, then that will be equal to the entire probability, okay, so what is a good U that you can choose, that you will know will be in all 0, okay, so what do you need to make the all 0 code word assumption in your calculation, all these probabilities given that a particular code word is transmitted have to be the same, okay, there are, one needs to prove this, okay, right, remember what are we doing in the decoder, we are doing something which is not necessarily the same for each received code word, right, if you receive a code, transmit a code word, the actual messages will differ, okay, some things might be 1, some things might be 0, everything will change, okay, so one needs to carefully prove it, okay, so the trick to proving this is to show, to show that if you transmit any code word U, okay, and then you face an error vector E, it is the same as transmitting 0 with the error vector E and then you can map these two decoders 1 to 1, okay, so that is what people do, people show, if you transmit U and face a particular error vector, okay, it is the same as transmitting 0 and facing maybe some of the error vector, I believe it is the same error vector, so you have a 1 to 1 correspondence between a decoder for a code word U and the decoder for the all-zero code word, they run exactly the same way, exactly the same way in the sense, if you have an erroneous message in one decoder, the exact same message will be erroneous in this other decoder for the all-zero code word, okay, so you do that mapping and then you show it works, okay, that needs to be done, it needs proof, okay, but we will not see that proof in class for, I will not do that proof in class, but I will just say that it is possible to show it, okay, a crucial necessity for this is symmetry of the channel, okay, usually you need the channel to be symmetric, okay, the decoder should also satisfy some conditions, on top of that the channel should be symmetric, symmetric in the sense what, channel should do the same thing to both 0 and 1, okay, for the BSE is a very classic nice symmetric channel, it does not do anything very different to 0 or 1, so it works out, okay, so channel symmetry helps in this proof, the decoder operation is also important and one can do that, okay, so by the way if you want rigorous proofs for many of these statements, there is one central reference, it is a book called Modern Coding Theory, okay, I believe at this point it is available for free download from Rudiger Erbank's website, this book was written by, it is being written I believe by Rudiger Erbank and Thomas Richardson, do you remember these names, okay, I mentioned these names before, okay, you can do a Google search, you will find it from Rudiger Erbank's website, okay, but it is a difficult book to read as in it is very rigorous and proper and they have done a lot of careful study, but if all these missing proofs that I keep saying that I will not do this proof, you can go to this book and look at the proofs, okay, it is a very, all the detailed proofs are available, available there, okay and go back and check and convince yourself that the all-zero code word assumption was very crucial for us in the simplification of the density evolution, otherwise you will be stuck with some messages being, see, we could happily say probability of error is the same as probability that UL is 1, okay, if I did not know the message was all-zero, I will have to be careful, it will depend on the actual bit and then I have to do all kinds of calculation, it will be the same, it will not change too much, I can show it to be the same, but the calculation, the computation becomes much easier if you do, okay, probability of, oh, it is looking at the error vector instead of looking at the thing itself, yeah, maybe, yeah, maybe, maybe, maybe, so it is a little bit more tricky than that, go and look at the proof, I mean you have to be careful with the steps of the, steps of your decoder, okay, you should not see the steps of the decoder should not depend on what exactly U was, I mean the code word was, right, so sometimes, it is not very clear that the way we described it, it does not depend on that, when you are saying the steps of the decoder should only depend on E, yeah, I agree with you, when you do a syndrome decoder, first thing you do is H times R transpose, which becomes H times E transpose, so anything you do is only dependent on the error vector, so you can happily assume all-zero and do all these things, you have to show that the steps of your decoder are getting rid of the syndrome, getting rid of U and are operating only on the error vector, it is not clear because you are not directly computing all the parity checks, you are doing only partial parity checks, right, you are doing, right, you remember the message from the check node to the bit node, it is not, it is not the full parity check, right, it is parity check XORed with one of the things, so you do not know a priori if you are only dealing with the error vector or the code word itself is involved, so you need to prove that, that is all, okay, all right, so we will stop here for today, in the next class we will pick up from here and continue so forth.