 Welcome to this lecture on digital communication using GNU radio. My name is Kumar Appaya and I belong to the department of electrical engineering IIT Bombay. In the previous set of lectures, we have taken a look at error control codes. In particular, we took a close look at the parity check code, the wherein the parity of the code word was forced to be even parity, meaning it contained an even number of ones. And we looked at the repetition code. So, if you take a particular bit and repeat it 3 times or 5 times, we saw that that resulted in a much lower probability of error on the binary symmetric channel. But the price we paid was that the parity check code could not correct errors, it could only detect a single error while the repetition code could correct errors, but it was very inefficient to use because of the low rate that it provided. To generalize our discussion on codes, in this lecture, we are going to take a look at linear block codes. And linear block codes are very commonly used error control mechanisms. And they form a building block for many practical error control codes as well. In particular, we will see the hamming code a little closely in further lectures. Let us now look at the definition of a linear block code. A linear block code requires two parameters to specify it n and k. So, you take a block of k bits that is you take your sequence of bits, take a block of k bits and then you map them to n bits where n is typically more than k using a binary linear transformation. The fact that you are performing a linear transformation is very powerful because it allows you to implement these operations effectively in software and hardware. The nomenclature we use, we have already been using this kind of nomenclature is that of n k binary code. And an n k binary code has rate k upon n. To implement these, we can use matrix techniques because it is a linear code. You will see that we can actually implement these using matrix multiplication with modulo 2 addition of course to implement the features. There are several useful results from linear algebra. For example, those are vector spaces that come in handy. While we will not discuss these in detail, I will mention some useful properties as and when required so that it becomes easy for you to understand. Finally, as mentioned, because of the fact that these are bits and you know operating on the field consisting of 0 and 1, the addition and multiplication operations are respectively exor and and therefore these are easy to implement using logical gates or even in software very very efficiently. So, let us just understand what a linear block code does. You have an n k linear block code that maps 2 power k k length binary sequences to 2 power k n length binary sequences. That is, if you take a block of k information bits, the total number of possible sequences that you can construct is 2 power k. These get mapped to 2 power k n length sequences where n is larger than equal to k, therefore there is redundancy added. For convenience, we denote the k length message sequence as a k size column vector m and the n length let us say n and the n length coded column vector as x. That is, you take the message express it as a column vector and the coded vector which is x is also an n length column vector. So, the k length column vector gives rise to an n length column vector after performing the coding operation. Now, naturally because we have this linear code, we can express this transformation as a linear transformation in the field of 0 and 1 once. That means, we can define a generator matrix G. The generator matrix G by convention however is defined in the transpose sense that is it has k rows and n columns. So, if you pre multiply by the vector m expressed as a row which is why I have written m transpose you get x transpose and the entries of G are 0s and 1s and each row is a code word. Let us understand what this transformation is doing. G is generally expressed in this way as a fat matrix meaning it has k rows and n columns. Each of the rows consists of something like a code word of this code because if you express if you take m as 1 0 0 0 or 0 1 0 0, it picks out that particular row each of those are code words because m can take 2 power k possible values, m transpose G can take a maximum of 2 power k possible values. Of course, if you take if you have some linearly dependent rows or things like that you may get less, but let us ignore that for now. So, if you carefully construct G you will get x for every m you will get an x and this x has enough redundancy. So, that even when there are errors you may be able to reconstruct them or you may be able to detect some errors that is the key. But let us understand this a little more practically before going into the actual mechanics of how you can use this generator matrix. Let us look at some examples that you have seen already. So, let us first start with our favorite examples the 3 2 parity check code and the 3 1 repetition code. So, let us look at these. So, if you look at the if you look at the parity check code we will actually write down all the elements of the parity check code it is 0 0 because we want even parity it is 0 0 0 0 1 1 1 0 1 1 1 1. Now, this code it turns out is a linear flow of course, a block code because you take blocks of 2 bits and map them to 3 bits we saw that in the previous lecture. But this is a linear block code why the reason is because we will show that you can construct a G matrix and use m's that is 2 length bit m's to generate all these code words. Let us see to construct G for this particular code word what we will do is the easiest way is to take some 2 linearly independent code words so to speak 0 0 0 is a trivial quantities you know you cannot really multiply it to get anything useful because if you multiply anything by 0 0 0 you get 0 0 0. So, let us actually define our G using any approaches I am going to take the second and third rows let us say I write it as 1 0 1 0 1 1 I took this one and this one. Let us now use the definition we have k is equal to 2. So, m is a 2 length column vector let us evaluate m transpose G for every m I am going to write m as rows so let us say 0 0 1 transpose G is what you multiply 0 times 1 0 1 and add 0 times 0 1 1 and you get 0 0 0. Next we multiply 0 1 if you take 0 1 what do you get it picks out the second row you get 0 1 1 if you take 1 0 it picks out the first row. So, you get 1 0 1 finally, if you take 1 1 it adds these 2 rows and you end up getting 1 1 0 now what do we conclude over here. So, there are many things the first the n length 0 vector is always a code word of any linear block code why because choosing m as 0 always results in the z n length 0 vector because that is a property of the linear transformation. Therefore, the n length 0 n length 0 code word is always an element of the linear block code that is the first thing. The second thing if you look at these these are the same as these in other words you are able to construct all your code words using this generator matrix G. This generator matrix G therefore, is a valid generator matrix for this code word ok great. Now, the mapping is 0 0 this is one mapping of you know m to m transpose G that is something which you have. The one other question which I have is of course, you saw that any linear combination of the rows of G is also a code word because you took 1 0 0 1 and 1 1 you got code words. Now, is this G unique where it turns out that this G is not necessarily unique because let us actually take a different G as an example. Let us take G to be for fun let us take it to be 1 0 1 and 1 1 0. You can show that you know these 2 are linearly dependent because you cannot just multiply the first and get the second ok. You cannot just add them up and get 0. So, these are linearly independent. Let us actually evaluate m and m transpose G for this again at 0 0 conveniently I am not writing the full form 0 1 1 0 1 1. If you take 0 0 you get 0 0 0 great. If you take 0 1 you get 1 1 0. If you take 1 0 you get 1 0 1 if you take 1 1 you get 1 1 1. If you in look at this enumeration you get 0 0 0 0 0 is here 0 for 0 1 I will get 1 1 1 yeah and 1 0 1 is fine and for 1 1 I get yeah ok. So, now if you enumerate if you look at compare the comparison 0 0 0 0 0 0 1 1 1 1 1 1 1 0 1 1 0 1 1 0 1 1. Now, this is another equivalent generator matrix for this code, but there is a slight difference look at the mapping from the messages to the coded vectors. In this case 0 1 became 0 1 1 this came 0 1 became 1 1 1 this came 1 0 became 1 0 1 1 0 remained at 1 0 1 1 1 was 1 1 0 here 1 1 is 0 1 1 here. So, the mapping between the messages and the coded vectors is different here. However, in terms of error performance or all other matrix these two codes are equivalent they are an equivalent code ok. So, you can actually come up with multiple equivalent realizations of the code this is something you have to bear in mind. So, for example, sometimes some people may say this is a 7 4 Hamming code and give you some generator matrix. I will then come up with 7 4 Hamming code and give you a different generator matrix which may have a different mapping nevertheless they are the same or equivalent code. That does not matter because the mapping between the messages and the mem transpose g's can be looked at as a table look up and you can always correct for it. So, couple of things to note are you can always just swap the rows and this is something I am not proving you also swap the columns of g without losing any performance the 0 vector is always a code word ok. Now, one thing to remember is when you construct the generator matrix you should take some linearly independent code words do not take 0 0 0 you will not get all the code words because 0 0 0 should not be a row of the g if you want a non trivial code ok. The next thing we are going to do is we are going to look at the n 1 let us take 3 1 ok. The generator matrix is very simple because if you have a 1 repetition code your m is a 1 element vector. So, g has to have only 1 row this is the only g that you can come up with because if you now take m is actually 0 or 1 since m is 0 or 1 m transpose g is either 0 0 0 or 1 1 1. So, for the n 1 repetition code the g is of this form ok. So, this is actually very very simple if you think about it. So, if I ask you for an n 1 repetition code the generator matrix is just a row consisting of n 1s because you are just going to multiply this by 0 or multiply this by 1 and you get the 2 code words because the n 1 code has 2 power 1 equal to 2 code words. So, the exercise is generate all code words and check the above g's we just did that and one other exercise is try to enumerate all possible generator matrices that are equivalent for this linear block code. Now, now one thing is that see how can we check whether the channel output is a valid code word. So, one thing you can do is you can go through all possible m transpose g's and then when the receiver you get any n block sequence you can keep checking whether it is one of those one of those one of the one of the code words if it is not a code word then you have to do something. But this kind of comparison with each code word is not efficient it is not the right thing to do because you have some interesting properties by virtue of this being a linear code. So, in other words whenever you receive an n bit sequence it is not a good idea to check this n bit sequence against all 2 power k code words to check which message it corresponded to. In fact, you should find out whether this is the code word by performing a linear operation the hint I am going to give you is that since the code words were generated by taking linear combinations of the rows of g all you need to do is to verify whether you are definitely in the linear combinations of the rows of g by doing by you know to do that you can always come up with a so called check and this kind of check is done by using a parity check matrix this parity check matrix is a pair matrix like g and h come in pairs whenever you generate your miss your coded word coded words there is code words using g there is a corresponding h that will check whether indeed this particular n bit vector is a code word of this code. What surprisingly one way to think about it is you have a total of 2 power n possible n length sequences only 2 power k of them are valid code words and these 2 power k form a vector space. Now what you can do is the remaining 2 power n minus k are not code words right. So, you can come up with a way to check this property using h. So, the h is a matrix of size n minus k cross n such that h m is 0 for all valid code words that is you need to come up with n length you know strings who which have 0 inner product with all the code words this is the one way to look at it. Another way to think about it is h times g transpose yields the 0 matrix ok. So, how does this work and why does this actually function well. Let us actually perform an exercise and try to intuitively guess the parity check matrices for the repetition code and our parity check code ok. So, let us look at the parity check code and we will take this particular version of g sorry ok we want. So, this in this case is a 3 2 code n is equal to 3 k is equal to 2. Now h is n minus k rows n columns n minus k is 1 this h ok should be 0 for all valid code words. So, in other words you want h times 0 0 0 is 0 that is fine h times 1 0 1 should be 0 h times 0 1 1 should be 0 and you want h times if you add this 1 1 0 to be 0. So, you want a parity check matrix that satisfies this. Now how will you get such a parity check matrix now we will try to guess ok. So, we let us let h see h is going to be a binary matrix. So, we can basically have entries a b c ok. Now what is the first equation tell you a plus c remember XOR is 0 ok. The second equation over here says b plus c is 0 and the third equation says a plus b is 0. So, we need to just come up with a b and c that satisfy this right. So, we need to come up with just a b and c that satisfy this condition ok. Now one way to do it is to guess ok. Now one guess I will make is that if I guess that h is equal to 1 1 1 does that satisfy this condition 1 plus 1 is 0 1 plus 1 is 0 in the binary sense 1 plus 1 is 0 1 plus 1 is 0. I am guessing that 1 1 1 is the parity check matrix. Let us check. So, remember our code words are I am ignoring 0 0 0 because that is always going to give you 0 this is 0 1 1 what do you get 0 plus 1 plus 1 0. If you do 1 0 1 sorry that gives you 0 great and do I need to check for 1 1 0 no because if I just add this 1 and this 1 I get 1 1 0 and I can check it or 1 1 0 is going to add the first 2 and I get it. So, in other words for the 3 2 parity check matrix 1 1 1 is the parity 3 2 parity check code 1 1 1 is the parity check matrix. So, h is equal to 1 1 1 is the parity check matrix. You can also verify that h times g transpose is 0. So, what is our g over here? Our g is 1 0 1 and 0 1 1. So, if you take 1 0 1 0 1 1 you get 1 times 1 plus 1 times 0 1 times 1 plus 1 times 0 plus 1 times 1 is 0 1 times 0 plus 1 times 1 plus 1 times 1 is 0. So, this indeed is the parity check matrix of the code ok. So, what size should this be? See this is n minus k cross n and this is g is originally k cross n cross k. So, this should be n minus k cross k 0 matrix and that is indeed the case. Let us actually repeat this for our repetition code. Let us say 1 1 1 we want a parity check matrix. For this right let us actually take the approach of finding a h such that g transpose for h g transpose is 0. So, n is equal to 3 k is equal to 1. So, my h has to be a n minus 1 2 cross 3 binary matrix right. So, I need a 2 cross 3 binary matrix. Now, I want h g transpose is 0 that means, h times 1 1 1 should be equal to what is the 0 size. So, remember this is n minus k cross n this is sorry this is n minus k cross n this is n cross k. So, this is n minus k cross k in our case it is 2 cross 1. So, we want 0 0 we want something like this ok. So, now how do we get it what h do I choose ok. Now, it turns out that the parity check code I chose earlier and the generally you know and the repetition code I am choosing they are very connected. So, if you just use the same approach you know to say I want to you know I will choose the entries of a b c d e f and things like that and work it out you will find that the h is actually the generator matrix of the parity check code there, but let us actually not do it that way let us do it using a more systematic approach. See if you want h g transpose to be 0 you need to know choose 2 rows such that their inner product with h r 0. So, I can always just guess. So, let us say that 1 0 1 and 0 1 1 are my rows. So, you can check that if I choose my h in this manner this will satisfy this condition 1 0 1 times 1 1 1 is 0 0 1 1 times 1 1 1 is 0 also this is not the unique h you can also choose 0 1 1 1 1 1 0 this will also work. So, once again you can verify that the h is not unique ok there are multiple equivalent h s all h s are equally you know usable you can always come up with linear combinations of the rows and come up with different h s. So, this it turns out is the parity check matrix for the code. How did I get this if you want to do it systematically for a small case you can always just write them as a b c d e f and then write down those equations you will get the equations which you can solve basically you will get something a plus b plus c is 0 d plus e plus f is 0 and you can always just substitute some values and guess that this is indeed the correct solution. So, now in this manner we are able to come up with parity check matrix and the reason it is n minus k cross n is because this n minus k is indicative of the vector space elements that are not in the k. So, it is like you are partitioning your 2 power n into 2 power k and 2 power n minus k that is the reason why it is you know n minus k cross n and in fact there is another result that says that these codes are duals of each other and there is a big literature around how you can use this geometry to good effect. Indutively we want to construct a parity check matrix and this parity check matrix is very very useful to check whether your code word is correct without having to do the comparison with all the g's all the m transpose g's. So, you can always just go through what we just did and we just did this what 3 length sequence yields 0 into 0 into product of 1, 0, 1 and 0, 1, 1 we found that it is 1, 1, 1 of course, 0, 0, 0 is trivial do not take that. If y is an n cross 1 received vector from the channel now here is the interesting part any 1 bit error will lead to h y is equal to 1 which is not 0 why because if you now look at the h where we chose 1, 1, 1 right it should have even parity right, but if only 1 bit is flipped 1, 0, 1, 0, 1, 1 or 1, 1, 0 or 0, 0, 1 any of these even if 1 bit is flipped if you multiply by this h 1, 1, 1 by the received code word you will get 1. So, the parity check matrix has a parity code has a parity check matrix if you multiply the received code word with this h if only 1 bit error has occurred you will always get 1. So, multiplying with the parity check matrix signal that a bit error has occurred. So, this indeed is an interesting way by which you can detect the errors of course, intuitively what you are doing is just exhoring the code words bits and checking that the parity is 0 if it is 1 then you can say that 1 bit has error has occurred of course, as I remember as you remember in the previous lecture if 2 bits 2 bit errors occurred or all 3 bits are flipped then of course this fails but still if only 1 bit error has occurred then this will correctly tell you that 1 bit error has occurred. Similarly, if you take the 3 1 repetition code as we just saw the generator matrix is 1, 1, 1 and this is one of the equivalent parity check matrices you can of course, check that all proper code words yield hx is equal to 0. So, 1 bit error directly yields the column with which the error has occurred how? See here is the interesting part if you take hs 101 and 0, 1, 1 if you then post multiply if you then post multiply by any code word. So, what are the code words? The code words are 0, 0, 0 and 1, 1, 1 ok let us actually just do this. So, this is the. So, if you take this if you take the parity check matrix in this manner and let us say that you send 0, 0, 0 but 1 bit error has occurred what do you get? You get 1 this picks out the middle column 0, 1. 0, 1 if you rate it in binary is 1. So, if you basically label this as 0 label this as 1 label this as 2 this indicates that a bit error has occurred in the second bit which is indeed the case and this is not magic let us try something like and let us say you sent 1, 1, 1 but the first bit was flipped ok 0, ok I am my labeling is probably incorrect, but it is ok maybe I will just change this change this labeling to this is 1, this is 2 and this is 3 yes because you did 0, 1, 1, 0 ok. So, now if you lose this approach 0, 1, 1 picks out the sum of picks out the sum of these 2 and I get 1, 0. 1, 0 indicates that the column corresponding to 1, 0 has an error which means the error is in this particular place and flipping it gives you the correct code word. So, in this manner using a repetition codes parity check matrix just by finding out the result 0, 1 that corresponds to the column where the error has occurred and that flipping that bit will give you a valid code word. So, this is something that will come in handy if you design your parity check code such that it has a nice parity check matrix finding out h times whatever it is received will give you the parity and will rather will give you the column or as combination of columns of the parity check matrix that will tell you the exact location where the error has occurred and you can flip that to get the code word and use that to find out which k bit message was sent. Therefore, this confirms that your repetition code can correct 1 bit error reliably. So, as we saw for these linear codes by multiplying by the received block of n bits you can easily find out whether you have a valid code word, but if it is not a valid code word most you know multiplying by h gives you a non-zero element and this non-zero element can tell you what the error is assuming that an error has occurred a detectable error has occurred. Let us take a closer look at the 3-1 repetition code as we just saw all possible single bit error vectors are 1 0 0 0 1 0 0 1 and with the respective h times the error being 1 0 0 1 or 1 1 this is for the parity check matrix that we saw earlier. If you take any of your code words x which is basically 0 0 0 or 1 1 1 written as a column vector I am writing it for convenience in this manner for an error pattern e with y is equal to x plus e. So, x is either 0 0 0 or 1 1 1. Now, one thing you saw that whether I sent 0 0 0 or 1 1 1 h times y always gave you the correct location of the error correct you know h times e y. The reason is because if you calculate s defined as h y s is also called syndrome h y is equal to h times x plus e and because this is a linear operation you can write it as h x plus h e. Now, you know that h x is 0 for a valid code word because when you multiply x when you multiply h by 0 0 0 or 1 1 1 you get 0. So, you get only h e as a result. Therefore, whenever you have any linear block code multiplying h by the parity check matrix gives you the same error pattern irrespective of the code word that you sent. In other words whether you send 0 0 0 or 1 1 1 if the error pattern is the same h times y will be the same value that is irrespective of what code word you choose the syndrome or h y depends only on the error pattern. This is the neat kind of result that we are going to use when we discuss single error correcting codes called hamming codes in the next lecture where we are going to use the syndrome to correct errors. So, to summarize in terms of any error correcting code, but in this lecture about block codes redundancy in the bits can help detect and correct errors and in particular we took the binary symmetric channel as our model and it is a simple, but practical model for handling errors in digital communication systems. As I discussed with you you can take BPSK over AWGN as one of the typical use cases of the binary symmetric channel, then we discussed the basic concepts of linear error correction and detection, linear error detection and correction and we generally basically gave some very simple example of linear block codes that can be used to detect and correct errors. Of course, we also discussed that whenever you take from k blocks to n blocks you are incurring an overhead. In fact, out of those n bits only k correspond to information bits and the n minus k are redundancy and this n minus k upon n is to be thought of as a redundancy. So, the rate of the code is k upon n, you want this to be as high as possible, but of course, you have to pay a price for the redundancy. In the next lecture we will continue our discussion on error control codes and in particular we will look closely at the so called hamming code. Thank you.