 Let's see this is lecture 2 and it's okay lecture 2 of error control codes. There you go. Good start. Okay, so let me again once again refresh us to where we were. We were looking at a linear encoder and what did we say a linear encoder would do? Would take a message m and multiply with a matrix G on the right to get a code word C. Okay, something very nasty has happened. All kinds of things are changing but anyway it's okay. I think I should rotate it like this. No, that's fine. Okay, so what are these different quantities? This quantity m is k bit vector and this is going to be a k by n matrix. This is going to be a n bit vector. Okay, so you actually want to send the message m and to give it some protection from errors, etc., you're going to add some redundancy of minus k bits. It's the parity bits and I enforce the constraint that each parity bit should be a linear combination binary XOR of some chosen subset of the message bits. So based on that we got this simple situation. Okay, so this is the first view of what actually happens in the linear encoder trying to generalize it a little bit. Can one of you go over there and tell those guys that's a class going on and move them a little bit outside? Thank you. So this is, this was the generator matrix view. Okay, so this is very important and this will nicely generalize as I briefly mentioned to the basis for a subspace which the code will actually be. Okay, once again remember the crucial definition was the code. What is the code? Code was the list of all code words distinct from the encoder which is a mapping from message to the code. Okay, so that was that's one more fact which I mentioned last. Okay, there's also what's called a parity check view. Okay, this is very interesting. Parity check view. Okay, so again I'll introduce this with the example and then we'll once again look at it as a matrix and we'll see how that works out always. Okay, so in the example I had what were my three encoding equations? I had p0 equals m0 plus m1. I had p1 equals m1 plus m2 and I had p2 equals m0 plus m2. Right, these were my three equations. Okay, so I'm going to play around with these equations a little bit and interpret them as checks that every code word has to satisfy to belong to the code. Okay, so here what have I done? Given a message I'm generating a code word from the code. Now I want to do something else which is closer to the receiving side. Suppose somebody gives me an arbitrary n-bit vector. I want to be able to tell whether it's a code word or not. Okay, so what so how do I go about doing it? For that you can again manipulate with the same equations and get those checks. Okay, what do I do? I simply move all the things that are on the right hand side to the left hand side. I'll get constraints that bits in the code word have to satisfy which are called parity checks. Okay, so let me move these things around. So I'm going to move m0 and m1 to the left hand side. So what will it be? p0 plus m0 plus m1. Why? Because everything is modulo 2, right? So I don't have to worry about anything else about any minus ones that might show up. Okay, so p0 plus m0 plus m1 equals 0, p1 plus m1 plus m2 equals 0. Bear with me if you've already seen what the exact thing should be but I want to write it down just to drive home the point that these two are completely equivalent. If you do any encoding according to the three equations that you see on your left, okay, and if you enforce the parity check constraints that you see on the right, you will get the same set of code words. Okay, so do you understand what I mean? Okay, suppose I give you a 3-bit message and find the 6-bit code word according to the equations on the left that you see here. Okay, I'll get a list of code words. Suppose I go through the list of all 6-bit vectors and only pick those vectors which satisfy the three equations on the right, I'll get another list of code words. These two lists will be exactly identical. Okay, this will follow very readily if you look at it very closely. You don't have to worry too much about it. Okay, now these equations, see we wrote the equations on the left in the generator matrix form, right? We're going to write the equations on the right also in a matrix form and that will be called the parity check matrix. Okay, so how do I write that now? I want a matrix H, okay, what's the parity check matrix now? Okay, I'll call it H, it's always called H in most places. H, okay, it satisfies this requirement. H times C transpose equals 0 for all code words C. Okay, so that is a matrix. So I want a matrix H and H times C transpose should satisfy that property. It should be, it should vanish, right? H times C transpose should be 0 for all code words C. Okay, so I have to say one more thing to actually fully define that. You should also say that H times C transpose should not be 0 if C is not a code word. Okay, I'm not explicitly written that down but it's okay. I mean, it's clear enough from the way I've written it down. If it becomes 0, for instance, if I pick H to be the all 0 matrix, it's going to be 0 always. It's no good to you as a parity check matrix. You want it to be 0 only if it is a code word. It's not a code word, it should not be 0. Okay, so it should clearly say that that's built into it. I'm not writing it down explicitly. Okay, this is the first thing. The other thing is what can you say, what more can you say about H now? H should have how many columns? Well, I don't know anything about the rows. I'm not saying anything about the rows but only thing I can figure out just because I'm able to multiply with C transpose on the right, what should it mean? H should have definitely n columns. That's the only thing I can say. I can't say anything about the rows. We'll come to it soon enough. Okay, but it should have n columns. Okay, that's pretty much the only thing you can say. Okay, so I want a matrix of this form. I should hide this. Okay, so I have a matrix of this form. Okay, so for this example that we had, let's try to cook up that matrix. It's very easy. In the example, okay, okay, okay, there you go. Alright, so in the example, it's very easy to cook up that matrix. Right, so I want some matrix on the left which will multiply my code word. What was my code word? M0, M1, M2. Remember, I have to transpose it. P0, P1, P2. And I should get, where did I put a bar below this 0? Okay, so I'm expecting this to be a vector, right? What will be the length of that vector? Number of rows of H. Okay, so it's very easy to see all those things. So I want, I'll put three 0s on the right because I know I'll get only three equations. Okay, so it can actually get more. Okay, right now we'll work with these three equations. Okay, alright, so what does the first equation tell me? P0 plus M0 plus M1 has to be 0. Okay, so I can put 1s here very easily. 1, 1, 0, 1, 0, 0. Is that fine? Okay, what about the second equation? It says M1 plus M2 plus P1 equals 0. Okay, so go back and write that down. 0, 1, 0. Alright, now what is the third equation tell me? M0 plus M2 plus P2 equals 0 and I can write that down very easily as 1, 0, 1, 0, 0. Okay, so there you go. I've formed a matrix H. Can I add one more row now? If I want to, suppose I want to add one more row, can I add one more row? Yes or no? Let me see, how many of you say no? Okay, how many of you say yes? Yeah, actually you can add, how many, you can add so many more. Okay, they'll not be independent. That's another point. I don't care if it's independent or not. Okay, you can add one more row. What, what can I add? I can do? Yeah, all 0. Okay, don't, don't give me very trivial answer. They can even add non-trivial rows. Right, what, what are the non-trivial rows I can add? Any linear combination of these three rows can be added and you'll still get another parity check matrix. It won't be the most reduced form but you can take linear combinations. Right, because I know that's satisfied. This check is satisfied. Take any linear combination. Again, it's going to be satisfied. It's no problem. Okay, so I could take a linear combination but these, these three have a special property. Okay, they are linearly independent and they fully define the checks. Okay, that will come to soon enough. Okay, but this is the parity check matrix for the code, for the example code. Okay, now it's not clear if this is readily possible in the general case also. Okay, let's, it's quite clear that it's possible in the general case. Right, if you have a, if you have any generator matrix. Right, what does the generator matrix tell you? It tells you how to write each parity check as a combination of several message bits. You can always move it to the other side and then convert it back into equations. For each parity, you'll get one equation. You should be able to do it. Okay, but it's nevertheless easy to have good if you have a general formula for how to go from G to H. Okay, so it's good to have that. Once you have that, you don't have to write all these equations out and move things from one side to the other. Maybe you'll lose something when you move. Right, so let's not, let's not do all that. So the general formula is also very, very simple. Okay, if, so in general, if you have a generator matrix of the form I, okay, I'll put a K here to qualify the, what am I, the size. Okay, when I say IK, it is a K by K identity matrix and then P. Okay, what's the size of P? K by N minus K. Okay, so that's clear. The parity check matrix you'll see if you write down the equations and do the moving around will turn out to be P transpose I N minus K. It's clear it should be N minus K, right? Why should you have N minus K? You'll have one equations per, one equation per parity that you generate. Right? See, each equation I got was for one parity. For every equation, I can move things to the other side and then write one equation. So for every parity, you'll get an equation. So you'll, in following that procedure, you'll get N minus K equations. Okay, that's the same important point. So you can always do this. Okay, so that's what is important. Alright, so now we know for any linear encoder that one comes up with in the systematic form, it's easy to go from generator matrix to parity check matrix. Okay, so all this is very, I mean, a lingo very close to coding theory. People think of generator matrix, parity check matrix, the rows of the generator matrix have some significance which we'll see soon enough but that's how the lingo goes. Okay, now what we'll do next is to interpret all this using the language of vector spaces. Okay, which will give us a very nice generalization. At the same time, you'll get a deeper understanding, hopefully, using your knowledge of vector spaces as to what these matrices are and what way should you think about them. Okay, so that's what we'll do next. So that's the vector space view. Yes. Yeah, that's true. So the question was the number of independent rows in edge will actually be equal to the number of parities that you generated. Yeah, it will be. The number of independent rows in the generator matrix will be equal to what? Number of message bits that you have. Okay, so the vector space view. So first thing we'll do is define the binary vector space. It's quite simple to see. So this set 0, 1, n is actually a n-dimensional vector space. Okay, is that a good thing to say if I say just n-dimensional vector space, does it mean anything? What else should I say? Over what field? Okay, it's very important, right? The vector space doesn't exist by itself. It's over some field. Okay, so I have to say what field it is. Okay, there are several fields over which it will not be a vector space. Okay, it's over. It's a vector space over what's called the binary field. Okay, we'll see that for now over this field 0, 1. Okay, so what is a field now? A field is something where you can do addition, multiplication, division and all that. Okay, so it's very clear if you have just two elements 0 and 1 that you can add and multiply and divide and subtract and do everything you want as long as you do it modulo 2. Okay, as long as you do everything modulo 2, you can do all those things. So 0, 1 actually becomes a field and this 0, 1, n is actually an n-dimensional vector space over 0, 1. Okay, so I will not labour the point too much here. Okay, but if you don't understand it too much, if you don't understand the theoretical details, great deal. Practically what's important is once you have a vector space, what is it that you can do? You can add two vectors and what should you get? You should definitely get another vector in the same space. Can you add two n-bit vectors? Yeah, just simply xor bitwise. Okay, so that's the important thing. So the addition in this vector space is bitwise xor or another way to think about it is each bit is added to the corresponding bit in the next vector modulo 2. You just do the regular addition but you reduce it mod2. That's the way to think about it. So another operation that's important in the vector space is what? Scalar multiplication. What are the scalars? The field gives you the set of scalars. Here the scalar multiplication is extremely trivial. Why? You either multiply by 0 or 1. Okay, what will happen if you multiply by 0? You'll get the all 0 vector. What will happen if you multiply by 1? You'll get the same thing. Okay, so I'm sorry, is that a question? Okay, so there's no real problem with this scalar multiplication. In fact, if you use more technical language, any group, any abelian group will be a vector space over 0, 1 because addition is the only thing that matters. So multiplication is quite trivial. It comes for free. So scalar multiplication is very trivial. I won't write down that very clearly. I'll simply say trivial scalar multiplication. So what's the good basis for this vector space? You can take the canonical basis. How will you take canonical basis? 1 followed by all 0, 0 1 followed by 0, 0 0 1 followed by 0. That's a good basis. Can there be any other basis? Yeah, you can have any number of, can you have any number of bases? It cannot be infinite. It'll be finite. The vector space itself is finite. There are only 2 power n vectors. So who can count the total number of different bases that you'll get? Can you count? If you cannot count, that's a good problem to work on. It's a nice counting. It's not very difficult to count. But still, if you've not thought about it, it's only a finite number of bases and it's possible to count it also. So that's some word about the basis. So we'll usually pick the canonical basis if we have any confusion. So you can have a basis. That's the next thing I want to write down. So the basis we'll typically choose will be the canonical basis. There's no problem with that. You can choose that. What else is interesting in a vector space? What else would you like? You'd like subspaces. Subspaces are definitely interesting. So yeah, this is a finite dimensional vector space. Obviously it has a subspace. How would you define subspaces? How would you go about generating subspaces for a vector space? The best way of doing it is just start with the basis for the subspaces. Provide any basis. If you want n dimension, you know that the subspaces should have dimensions less than or equal to n. Simply provide basis vectors. Suppose I give you m basis vectors. Then you know all linear combinations of those m basis vectors would belong to the subspace generated by that. All this is very standard. I'm going through very fast but I hope this is very basic review for you. I don't have to go through great detail. There's one thing you have to be careful about when you define subspaces. You can define subspaces in the most easiest ways to provide what's called a spanning set. Just provide a set of vectors and say the span of this set of vectors is my subspace. What is the basis for the subspace? It's an independent spanning set. So from the spanning set you have to throw away the dependent vectors and get your basis. And the dimension of the subspace is the number of vectors you get that way in your basis. That's the way you think about subspaces. So subspaces are fine. You can define them using basis. So the next thing which is most interesting in most vector spaces is a inner product. So the inner product makes a real difference in the way you understand it. So suppose you think of R3. R3 is a nice vector space. When you think of defining a subspace, what are the two dimensional subspaces of R3? The planes that pass through the origin. The planes have to pass through the origin. Otherwise it's not a subspace. So it's not a subspace if it doesn't pass through the origin. So how do you define a plane that passes through the origin? There are two ways of defining it. What are the two ways that you have learned in defining such planes? One is to provide two linearly independent vectors on the plane. What's another way? The normal. So how did you get the normal? Normal came from the definition of the inner product. The normal is going to be orthogonal to all the vectors on the plane. So the inner product plays a very good role in your understanding of what these subspaces are. Instead of saying what lies on the subspace, you say what is orthogonal to the subspace and that gives you a lot of information about what the subspace is, even though you are defining the orthogonal part. So the inner product is very important. Even in this binary vector space, there is an inner product which behaves very differently from your regular inner product and Rn in the real vector space. There are some very interesting properties that this has which makes it which makes the binary vector space different from Rn for instance. So let's define the inner product. I'll take x dot y. This will be my notation. Sometimes I'll use the, sometimes I might use this notation also in case the dot is a little bit ambiguous. I might use this notation. What is the definition here? x1 y1 plus x2 y2 plus so on till xn yn. But what? What is my plus? Modulo 2. Why? The inner product is always defined to take you from two vectors to what? To the field of scalars. You can't go anywhere you want to the inner product. So you have to reduce it. Modulo 2 and that Modulo 2 makes all the difference you need between the Rn and 01n. It makes a big difference. This is the inner product. All right. So the inner product seems simple enough. So hopefully all the intuition you have about from the inner product and Rn should help you a little bit. Many of the results are true but there is one result which is starkingly different in the binary case. So I'll highlight what is different. This is the difference. A difference from Rn is what? In Rn if x dot x is 0 implies what? Implies x is 0. Here it does not imply that x is 0. Why? Because you did Modulo 2. It's very easy to come up with examples of a non-zero vector which when taken a dot product with itself will give you 0. What is a good example? 1 1 0 0 0. In fact you can characterize all the non-zero vectors which will give you a dot product with itself as being 0. What would you characterize that? Even number of 1s. You take any vector with the even number of 1 1s in it then if you take a dot product with itself you will definitely get 0. So there are several of them. Exactly how many even numbered will be there in 0, 1, n? God, thought this was a very simple question. So how many even numbered vectors will be there in the set of all n bit vectors? Half. Does it make sense that it should be half? Convince yourself that it should be half. You'll have 2 power n minus 1 even number, 2 power n minus 1 odd number. So that's the inner product. So even though this is not true the subspaces for subspaces you can have dual spaces that all that will still hold. So you can go back and look at your proofs of dual spaces in vector space theory if you want very closely and convince yourself that for even though this is not true you can always define a dual space for a given subspace. Suppose I have so let me define that. Suppose I say s is the m dimensional subspace of 0, 1, n. What does that mean? If you want to be very specific about it, s should have a basis with how many independent vectors in it? m independent vectors in it. That's what specifically it means. For that subspace I can define a dual. So I'll call s perp. That's how you read that. s with the perpendicular sign on top, s perp. This will be the set of all vectors x in 0, 1, n such that x in 0, 1, n such that what? x dot y is 0 for all y in s. I can make this definition. There is nothing wrong with this definition. It works whether or not the dot product satisfies the norm condition which is x dot x not being 0. This will definitely work. There is no problem there. So that notation is for all. If you have not seen it before, it's short for for all y in s. So like you did before, you can show this s perp will also be a subspace. And you can on top of that show the dimension of s perp will be n minus m. All that will follow from basic linear algebra and Gaussian elimination. So the inner product and its properties don't really play into this picture. So all those things will follow from that. So you can show s perp is in fact a subspace of dimension n minus these two you can show. So the crucial difference between subspaces in the binary vector space and subspaces in Rn is in Rn where will s and s perp intersect? Only in 0. In the binary vector space s and s perp can intersect in a very non-trivial fashion. You can have a, in fact you can have s equals s perp. That's a very active research area encoding. How to find subspaces s in binary vector space so that s equals s perp. So you call self-dual codes if you've heard about them before. So it's an interesting thing to have. So that's a difference in binary. The crucial difference from Rn is s intersect s perp can be non-trivial. Just to increase your curiosity in it I can tell you s equals s perp is possible. All right. So any questions, basic questions. So this is a quick review. I mainly pointed out the difference between what you can expect in binary and real vector spaces without going into any great detail of or proof or anything like that. Wherever I need I will use additional facts. For instance, if I have a m dimensional subspace s and if I give you m linearly independent vectors, you can do Gaussian elimination and reduce them to what form? To a row reduced echelon form. And you will get some kind of a canonical system. You will have this non-overlapping form with this i, n, p-type structure. So that's how we went to the systematic form. All that they'll use whenever necessary. I'll assume you're familiar with that. It's no point in proving those things in a class encoding theorem. Any other question, curious point about this intersection? We'll see an example soon enough. That point maybe this will be clear. Let's see. Let's see an example. What n should I pick? What n do you like really? Which is your lucky number? Don't give me too large an n. I'm going to say 5. 5 is a good enough number to pick. So 5 is a little nasty as in you can't have this self-dual stuff. So let me pick a different number. So like the closest number, 6. 6 you can have some nice nice. All right. So what's the, so if I write down the all the vectors of the six-dimensional binary vector space, how many vectors will I get? 2 power 6 which is 64 vectors. So I'm going to define a subspace here, which is let's say, so here's more notation. So this is again notation which you should get comfortable with. If I say a subspace is this, let's say 1 0 0 0 1, then 1 0 0 0 1 0. Suppose I say this, what does it mean? What's the most obvious way of interpreting it? I'm not going to say this is the basis. I'll say this is a spanning set. What do I mean by a spanning set? You have to take all linear combinations of these two vectors to get all the vectors from the subspace. Suppose I call this as u1 and suppose I call this as u2. How many vectors do I have? Two vectors in my spanning set. So how many linear combinations can I take? I can multiply u1 with some constant and then add it to, right? So this s actually is what? a times u1 plus b times u2. What are a and b? a and b are just 0 and 1. So you can't have, so that will be a difference between these infinite vector spaces that you're used to in the finite vector space. Everything is finite. The total number of vectors itself is only finite. So you can't have these infinite notions, lines and planes going all over the place. It's just all finite. There's only a finite number of them. So if you do that, there are only four possibilities for a and b and you can easily substitute that and get all the four vectors that you'll have in this subspace. I'm not claiming u1 and u2 are linearly independent. In this case, are they linearly independent? Yeah, you can easily see that they're linearly independent. So it will also be a basis. In general, I might give only a spanning set and you should be able to deal with that. So if you do this, you'll get first vector you'll always get is what? All zeros. And then these u's will repeat. It's not a problem. And then the non-trivial vector you'll get will be 0, 1. Is that fine? Good. So that's how we define the subspace. So that's fine. So now if you go back to our generator matrix, if you go back to our generator matrix, what did we have? We had the code, the code word c being m times g. And m was what? All possible? K-bit vectors, messages. K-bit vectors. All possible? K-bit vectors is what we took. So now you see the connection between the way subspaces are defined and where you do this generator matrix. So if I have to write down the code itself, what is the code? Code is set of all c equals mg and then you take m belonging to what? 0, 1 k. That is your code. Set of all code words. So now if I imagine g is having k rows and n columns and suppose let's say these k rows are linearly independent. Just for now, I'll assume they're linearly independent. If they're not linearly independent, what will you do? You can do elimination and throw away the linearly dependent ones and get only the linearly independent ones. So I'll always say rows and g are linearly independent. In the systematic form that we picked, Ip, the rows are always linearly independent. So now we can always say they're linearly independent. I'll say i here, ik here and then p here. So let's write down this form in a different way. I'll say the first row is g1, second row is g2. I'm simply writing it row by row, g2 so on till the kth row. Suppose my generator matrix, the rows of my generator matrix are g1 through gk. So you see the connection is really clear. I'll write it down in the next page. So you see any code word c is actually m0 times g1 plus m1 times g2 plus so on till mk minus 1 times gk. And these mi's are actually bits, 0, 1. So what am I doing when I multiply my message with the generator matrix? I'm actually taking a linear combination of the rows of the generator matrix. And what is my code? The set of all possible linear combinations. So clearly the code is nothing but a subspace with spanning set equal to the rows of the generator matrix. So that clearly follows from here. The code is subspace g1, g2, gk. That's very clear. And if g1 to gk happens to be linearly independent, like in the case that we had before IP, then you know it's the basis also. So now you see there is an obvious generalization to the generator matrix. You no longer have to need it to be in the form IP. You can take any k vectors as the rows of the generator matrix. How will you reduce it to the IP form? You do those in elimination. So this is actually the most general definition for a linear code. Well, linear binary code, of course. But the way I defined it, for instance, I defined actually a systematic encoder for a binary linear code. So if you take the books, for instance, you'll never see the binary linear code being defined in that fashion. Binary linear code will always be defined as a subspace of 0, 1, n. That's the way you have to think about it. Now you notice one simple fact about the linear code which will be true. If you take any two code words and add them, what should you get? You should get another code word. I could have proved it starting with c equals mg. How will I prove it starting with c equals mg? Take m1g, m2g and then add what will happen. You'll get m1 plus m2 into g. m1 plus m2 is another message. So obviously, m1 plus m2 into g will also be another code word. All that you can do with that definition. But the moment you see it's a subspace, you feel better. So you have a nice abstract understanding of it. You say it's a subspace, you add any two code words you should get another code word. So all this seems, it just comes very naturally. So you see that's the general definition. And then from here, you again get that sum of two code words is another code word. All these simple properties follow very nicely from it. Another advantage in having this nice vector space view is, suppose somebody says, somebody gives you a spanning set which is not in reduced form. Then using your vector space knowledge, you'll know how to deal with it. Suppose I give you a generator matrix g. Suppose I define a code with the generator matrix g which is not systematic. It is not already in that form. What's the advantage in having it in systematic form? You know how to build the encoder. You know how to write a program for the encoder. You know how to form the parity bit as a linear combination of the message bits. If I don't give you in that form, what happens? It's not in systematic form, but you can still write a program for the finding the code word. How will you do that? You take simply do m times g, right? Simply do the linear combinations, you'll get that. There's not a lot of advantage to that. So there is some advantage to repeating the message by itself. Then only computing the parities as opposed to computing every bit of your code word. So if you want that advantage in your encoder, you'll have to do some work. What is the work that you have to do? You have to do a Gaussian elimination, linearly do all the elimination, reduce it to this form. But what are you guaranteed when you do all that? What does not change? The set of code words. So the code does not change. Why? You know from your elimination theory that all your Gaussian elimination does not change the row space of the matrix. So that's the advantage in having this vector space view. The moment you have this vector space view, you're not afraid of playing around with the code words of the code. You know, you can take all these linear combinations and as long as you do it properly and as long as you don't introduce any dependence or anything, your code will not change. Everything is fine and you have that comfort extra degree of comfort. Is that clear? So that's the advantage of going to this abstract view. At one point it might seem like totally unnecessary, but in practice it helps in being able to play around with these code words and come up with some simplified cases. So I'm going to stop here in the next class. We'll take a general, take a few examples of generator matrices, play around with a reduced to systematic form and then we'll slowly move into the parity check matrix side. Okay? Yes. Any k vectors will can span, can span a linear code. Yes. A k dimensional linear code. What do you mean by span? What do you mean by? I don't understand. See every set of k will not give you the same set of code words. You'll get the different code, but it is defined to be the code. No problem. You'll get a different code if you change your set of vectors. Well, you could get a different code if you change it properly. Is that your question? What is your question? But it may not be a good code. I'm not saying it's a useful code. I'm not saying it's practical or you want to use it. It is a code. The definition is a code.