 So this is the first lecture so it will be mostly introductory but we will begin in right earnest you know I mean we will just start you will see the space at which it goes that will also help you understand the rigor in this course. So to begin with error control codes are part of digital communication systems right. So how does a digital communication system look like communication system. So you have various ways of drawing this picture you go and talk to a field engineer or an analog engineer or someone to build systems he will draw you several blocks starting from all kinds of things to timing recovery etc etc ok. So I am going to draw the box that is used by people who do coding theory ok. So there are several boxes the box that we will pick is something extremely simple. I will say bits are going in ok and then I have a box ok and out of that box what comes out bits ok. So that is my beautiful digital communication system ok there is absolutely no information here. I am going to call everything that is in between as a channel ok. So we will work with this simple black box to begin with ok we will as we go along in the course we will change this we will go deep into this not very deep but at least dig into this channel a little bit and do something better ok. But for starting off we will start off with this very simple looking picture where everything that happens to a bit as it goes through a digital communication system and it is received at the other end is just one box and you get a bit out ok and we will in fact take a very simple model for that box we will simply say it is what is called a what do you think I am going to say now it is going to be a binary symmetric channel ok. So I will say it is a binary symmetric channel ok. What does this do it is very simple it takes a 0 ok and maps it to a 0 or a 1 and it takes a 1 and maps it to a 1 or a 0 ok and there are probabilities for these things 0 will become a 0 with probability p 1 minus p I am sorry 1 will become a 1 with probability 1 minus p and a transition or a wrong or an error will happen with probability p ok it is a very simple model ok. So that is a that is that is where we are going to start we are going to say anything that happens to the bit looks like this ok if you transmitted a 0 you will receive a 1 with probability what p you transmitted a 1 you will receive a 0 with probability p everything is symmetric whether you do 0 or 1 and channel is of course binary and there will be another assumption each bit what what happens to each bit is independent of whatever happened to any other bit ok that is just an implied assumption I am not going to write it down or make make it more precise I am going to say each bit goes through an independent version of this channel ok it is all probability p ok I think you must have seen this before this is a very simple channel today ok so what is the goal when you design a digital communication system what is the goal what would you what would you like this p to be people are giving wonderful engineering answers what is the ideal value for p 0 ok ok but whatever you do you will never get p equals 0 ok and in many cases it is very bad to just design a communication system to optimize this p for one bit to go down to a very very low value ok so if you try that you will see me later on also maybe we will prove it but not not to any great extent you it will turn out that you will need a lot of power in the transmitter to drive this p to 0 ok so it is not a very smart thing to do it is not a very efficient thing to do ok so that is the first remark that you should have learned from digital communication ok not efficient to drive p to 0 ok so roughly if you remember from your digital communications what kind of SNR do you need on an AWGN channel to drive this p to 0 close to 0 ok depends on transmitting constellations suppose they say BPSK ok what kind of SNR do you need for 10 power minus 6 BER on a AWGN channel with BPSK ok so that is what you should have learnt in digital communication right so roughly the number will come to 11 dB or something like that some such number ok some 11 dB 10 dB ok so it is very large what is the but there is some theory which says it is enough if you have what what kind of SNR do you need for sending one bit across without error from information theory you can learn that it is just 1 dB or so ok so you have a 10 dB gap between what you need raw power in BPSK and what is actually possible from information theoretic notions ok if you are not familiar with this but at least at least know that that is possible ok so it is possible to use much lower power to get close to 0 probability of error as opposed to very large power ok so that is where that is the role that coding plays in digital communication systems it can try to reduce this p in an indirect way what do you do you do not try to reduce this p but you do something else right that is where coding comes ok so what am I going to do I am going to try and do coding in this digital communication system ok so I will say efficient answer is provided by error control codes efficient solution you have to use coding ok so so people knew that you have to do coding more than 50 years back ok and currently I think the theory has developed to a stage where people also know what the best way what the best way of doing coding is also so one can say in many ways that the problem has been solved ok initially 50 years back people didn't know how to do coding now you pretty much know what you have to do ok so that is a that is the interesting alright so now so what is the system look with coding ok suppose you look at coding this is what you would do instead of sending one bit at a time into this channel you would collect some k bits ok and then you would do something to it ok I am going to call it encoding ok I will call an encoder put an encoder here and out of the encoder comes n bits ok and I am going to let n be greater than k ok ok so we will come back to this box soon enough but let me complete this whole picture then after this you have a binary symmetric channel say with probability p and then out of this comes what when n bits going to this what would come out n bits again ok not necessarily the same n bits that you will transmit and depending on n that will almost always happen it will never be the same ok and now one can presume that you can do something which we call which we assign which we call decoding to get your say k bits back one can loosely say this is what happens ok so this is how you would actually do coding across the digital communication system that I had for you before ok very simple basic way of looking at it ok so what has happened here several things have happened ok so we have to look at it very closely let us begin with the encoding case it is a very simple thing to look at the encoding ok so what this is called the message this set of k bits is called the message after encoding you get what is called a code word ok after the channel you get what is called a received word after the decoder you get what is called the decoded message ok so first thing I am going to do is I am going to write down what type of values each of these things can take ok so I am going to use some pseudo mathematical notation hopefully you will follow it will be very important ok what values can the message take ok so typically assume all messages are possible right if you say k bits the message is coming out of some source you assume the source has got good entropy etcetera and all all messages are possible so if you say k bits then I will have to plan for the message being any one of 2 power k possible binary sequences so I am going to say the message belongs to ok I will use this notation 0 comma 1 what can I write to the power k ok so this will be a notation for what what does this notation stand for ok all binary sequences of length k right ok ok suppose instead of 0 1 if I put 0 1 2 to the power k what does this mean? Alternary sequence simple notation ok so that is fine alright so what about the code word now ok now I have to say a few more things about the encoder ok I am going to say now my encoder is a is a 1 to 1 map ok I am going to say that next ok so once I say that once I say my encoder is 1 to 1 what do I know about the code words what kind of values will the code words take ok exactly so that is the difference between the message and the code word right so you have message belonging to 0 1 to the park k code word belonging to 0 1 power n but what is the difference all n bit sequences are not possible as code words cannot be code words why because I said the encoder was a 1 to 1 map ok which means there are only 2 power k possible inputs obviously only 2 power k possible code words must be there ok so you should note that ok so set of all messages here equals 0 1 k set of all code words here will not equal 0 1 n it will be some subset of 0 1 n ok it is a very simple idea look let us move over to the other side the received word side ok once again I can definitely say this am I right but what will be the distinction between the code words and the received words all are possible even if you say only 2 power k code words were transmitted right all received words are possible according to my channel model in fact if I only transmit one vector right with positive probability every vector in 0 1 n is definitely possible right all those branches are possible every bit can be an error or not be an error right so you will get all the possibilities with non-zero probability ok so that is the next thing to keep in mind so what is the task of the decoder now ok suppose the channel was not there what would the decoder do suppose I knew the channel did not introduce any error what would the decoder do exactly so this encoder did a 1 to 1 map it would do a inverse of that map you know it is a 1 to 1 map so there is a inverse but what has happened from message to received word is that a 1 to 1 map clearly not right so even though the code word was a 1 to 1 from code word to received word it became a 1 to many map actually in fact it became probabilistic map right so you do not know exactly which was which it went to ok so obviously the decoder cannot do the inverse of this map it has to do something more ok and it will turn out clearly the decoder can never be always successful with this model right decoder will definitely make an error right so our objective will be what to minimize that error ok objective in designing the decoder will be minimize try to minimize well we will do it in various ways probability that decoded message is not equal to transmitted message ok so that is the first point to keep in mind you can never have a decoder which will make which will have zero probability of error over the binary symmetric channel obviously it will have an error ok so to begin with in this course we will we will still insist the decoder will be a proper function ok so what it will do is if you give a certain received word obviously it should give a certain message ok that map will be there so the decoder becomes actually a many to one map ok ok so if I want to be more precise ok so if I want to be much more precise about exactly what this map is right so if I want to be more specific I should exactly do what I should specify what the encoder map is and what the decoder map is right suppose there is an engineer who wants to write a C program to implement your wonderful digital communication system with an error control code what should you give him you should give him the one to one map at the encoder and the many to one map at the receiver ok you have to do that ok hopefully you designed all that very carefully and your probability of error is very low ok so that is the that is the overall picture so in this course we will see basically the design of the encoder and decoder ok first of all why is the design of the encoder a non-trivial problem ok there should be various possibilities right when then when do you have a non-trivial design problem when there are millions of possibilities and you have to pick one among them right that is when the design problem becomes very non-trivial so let me see how quickly you can count ok how many possibilities are there for the encoder for a given k and n ok do people agree with their answer how many how many different mappings are there I mean it needs to be it needs to be a one to one map right right so I have 2 power k code words to select from 2 power n possibilities and how many ways can I do that 2 power nc 2 power k you might say ok well yeah maybe one can argue that the order will not matter for for the probability of error etc maybe maybe one can argue the order will not matter ok so but the absolute code words also actually they won't matter several of them you can throw but let's just say 2 power n p 2 power k if you want to be very precise ok so it's a you might say ok it's only 2 maybe it's a small number but eventually you'll see you'll have to push this n and k to be very large numbers you'll have to pick k equals thousands ok then n becomes also greater than a thousand right so if you do 2 power 2000 choose 2 power 1000 what do you think you'll get ok you'll get numbers which are astronomically huge more than astronomically you can't you can't deal with them ok so out of all those possibilities for the encoders you will learn in this course how to pick that encoder which will minimize the probability of error ok isn't that amazing right that's the way to think about it ok so the design issue here is selection of encoder first ok so that seems to be the obvious design question but let's dig a little deeper here ok I'm going to argue the encoder itself does not matter only the list of code words matter ok the same question about this ordering that people asked ok the encoder is what is the map from message to the code word but look at what's happening the message to the code word map is very can be arbitrary I can assign any message I want to any code word what is received is actually what is received when the code word is transmitted and not when the message is transmitted ok it doesn't really matter what what mapping I do we'll make that argument more precise as we go along but you'll see actually the selection of the list of code words is the real problem and not necessarily the selection of the encoder ok in practice the encoder is also very important the guy has to write a c program right if you give him a list of two part 1000 table and then ask him to write it down he's not going to be able to write as well the selection of the encoder is also crucial in practice but in theory to minimize the probability of error what's interesting is the list of code words are not necessarily the mapping from the message to the code word ok so you'll see but but the encoder is very crucial ok and then encoder ok well encoder itself is a non trivial enough problem and come out to the decoder ok for each encoder ok what is suppose you fix the encoder suppose you did some magic and forgot all the best encoder that you could get ok for that encoder how many different decoders can you have what's the decoder anyway just a many two one mapping from 0 1 n 2 0 1 k right how many different mappings can you have there ok that's true but anyway let's so people are people are being very smart about answering the question they're saying for a particular encoder you know already what the best decoder will be and why don't you go and do that ok I'll come to that later supposing you didn't know about all this ML stuff ok just looking at this from numbers point of view I can assign it's not required to be 1 2 1 or anything so to each received word I can assign an arbitrary message you don't have to do anything else right do you agree yeah maybe you want some constraints like that later on you can be smart about it but suppose being very dumb you see the number decoders is very huge ok so then design of the decoder also becomes very very crucial ok right so so you have to select the list of code words then you have to design an encoder for it and then for that chosen list of code words you'll have to design the decoder ok so both of these you have to do to be able to come up with a good coding system for your digital communication system is that clear ok anything else that I'm missing out here ok alright so that's the very high level picture of what what you have to do but but the next important and crucial question is is this even a good idea right why would you want to do something like this ok your the justification I gave you was I can drive the probability of error to a very low number ok but what what is the price I've paid in doing that what price have I paid I'm sorry ok so I think a lot of people people know a lot of these things we're giving more advanced answers ok at a very basic level the penalty you've paid number one penalties are first is complexity right from a very trivial system design point of view the complexity has gone through the roof right previously you could send one bit receive one bit and forget about the rest right there's no problem now it has become much more complex that's one ok and other this other thing which is more crucial in many systems is is delay ok why are we paying in terms of delay because we need exactly how to wait for n bridge delay ok so the answer is you have to wait for n bridge before you can decode one bit ok so so even if you're transmitting at a very high rate that delay can be a bit of a problem ok so in many real communication systems you see delay is a problem but you can deal with it if it's tolerable you can deal with it several things will change in that time constant you just play around with it you can deal with delay also ok but complexity is one thing and the next price you pay is delay ok but today I have to say with the advances in VLSI and all these things complexity is almost trivial you know I mean you can build huge chips with lots of gates and lots of computations so complexity is ok delay is still an issue people are people always fight to deal with it it can be dealt with it's not it's not an impossible issue but delay is an issue ok there's also a more crucial issue from a pure communication theory point of view information theory point of view which is this redundancy that you're talking about ok previously you used the channel once to attempt to communicate one bit now what's happening that's not what's happening what's happening you're using the channel n times to attempt to communicate only k bits ok you see why I'm saying to attempt to communicate why am I saying to attempt to communicate yeah there are errors right so even the even in the previous case though you attempted to communicate one bit in one channel use you were hardly ever getting one bit right you'll you'll have a lot of errors ok so that's that's very important that's the rate penalty ok so what is the rate ok the rate we will define as k by n ok so typically if you're used to rates of kilo bits per second megabits per second you have to read k by n very differently ok k by n is what it's a number of bits sent in one channel use ok that's the best way of thinking about it or if you want if you're a very much a VLSI person you can say k by n needs to be multiplied by the clock rate at which bits are being clocked into the channel before you get the actual data rate ok information data rate ok so this is this is very very fundamental ok so I'm going to try highlighting for for fun here let's let's try a let's try a green highlight it's beautiful ok so that's very crucial ok so rate is is very very very very crucial ok so other two will we'll just simply wish it away we'll say we won't deal with it in this course but we will deal with the penalty that you want to pay so what's ideal ideally you would like the rate to be very very high ok so so that's that's something you'll see ok so what have we gained what are what are our gains ok so I'm going to go to the next page to talk about our gains ok so the penalties we saw before what have we gained have we gained anything ok we don't know yet I'm going to hope to show to you that we will gain in probability of error ok basically that will be the that will be the gain ok probability of error will go down much faster ok when you do coding as opposed to not doing coding ok but there are a few other issues at the system level that you should be aware of ok you'll see eventually hopefully after half the course we'll see this ok you'll see you'll gain in transmit power ok at same error rate ok so let me qualify this at same transmit power ok you'll get both these gains in a real system and it's and the and the first point might be very obvious to you the second point is very very crucial ok in most real systems that is how it is really really used ok so you can send a particular data rate at a lower transmit power ok and most rs communication systems that are out there today minus communication systems what do you think is the most expensive part in the chain the whole transmitting chain and receiving chain what's the most expensive part sorry the power amplifiers right power amplifiers are the most expensive parts in any rf design ok why why is that so I think you've learned enough electronics you should be able to answer that question why is why are the power amps so expensive they have to be linear over a large power range ok so this bandwidth of these systems are also large ok those large bandwidth they have to be linear over a large power range ok so that's very very difficult to build it's not very easy to build those rf amplifiers it's very very expensive and if you go to somebody and say it's enough if you design it design a power amp for 3db lower it'll sell it'll save like I don't know crores of rupees ok it's a huge difference 3db might seem like a small number to you it's a huge number for in those kind of ranges ok so it makes a big difference real difference to systems ok so many people use coding to decrease the transmit power at the same error rate ok so as I said based on information theory it turns out you can decrease your transmit power by as much as 10db ok if you say 10db that's it I mean that's really you bring down your costs by a huge amount ok so error control coach today play play such a huge role in almost every communication system nobody really takes a system seriously unless there is coding in it ok so if you look at any one of your systems today so give you some practical examples I like giving two examples uh usually just to drive home the point as to how how useful error control codes are ok so let's see some examples ok cm still having trouble with this ok so the first example I usually give of a real system where coding you can see in your in your own experience it would have made a difference to you and in the real system so following ok so so I do this in every every uh coding class or lecture I've given hopefully people have not seen this before you've seen this before please be quiet ok so what is this ok there's nothing here I have to draw one more ok what is this it's a very bad picture but other than that what do you think this is ok come on man you guys have to be creative tell me what that is ok maybe I should make it more symmetric uh it's a bad looking picture let me draw it again ok I think there is a way to draw nice circles in this but I'm going to draw one circle and then there's a concentric circle around it what does that remind you of it's targeted for a while use your creativity what is it think of it I mean it's an example of a digital communication something I'm going to give you an example come on man huh wow there you go so it's never failed you know I mean I've tried this five times so far if somebody has guessed cd ok so I guess that tells you so that's the picture of a cd ok I'm sure it's a better picture better pictures are out there cd dvd whatever you want to think ok so what's the for the system level what's the issue you would like to put say a movie ok so some six hour movie if you want you want to put that inside the cd convert it into bits in some fashion say some methods and then you put it in there ok but what happens when you use the cd in practice you're going to get scratches ok so roughly how to think of the way to think of bits on a cdr pit and no pit ok so either a small hole that's small hole that's dug in the on the surface or it's not there and the laser is going to scan it then figure out where where there's a hole and where there's not there's a pit and then there's no pit ok that's what it's going to do when there's a scratch what happens the whole pit no pit thing goes away right ok so then you can't read it so you see situation like this error control codes are play a very different kind of a role which is interesting ok so you might say how does how do these gains that I said transmit power probability of error play into a situation like this there's a much more basic role of what's played by error control codes if you don't have error control codes on CDs it would simply not work right forget about transmit power anything at all ok turns out all those things are also there in the cd ok they gain on all that also but on top of that there is a very basic enabling device right error control codes otherwise you'll never do it when there's a scratch what will happen you can't play our favorite movies on the DVD right how many of you have had CDs being rejected because it can't be read right have you have you seen this situation a CD will play in some drives it won't play in some other drives ok so all those things are cases where people have designed different things so that the sonar is just enough sonar is not enough all those things come into the picture ok but recovering from scratches is essentially because of because of error control codes ok ok so think of the penalty you're paying here ok so what's the penalty you're paying you thought of rate there right so what would happen here if I used a very low rate code what would happen here space space becomes an important right number of bits you can put per square inch ok so you'll see the cd industry will define that as a big number number of information bits you can put per square inch you'll see the actual coding that's done there is very low rate actually but still information bits itself is quite large ok so for instance instead of if you think of a movie like Lagan or something instead of putting it in one DVD you'll need probably some 10 DVDs if you use a very low rate code right and that's a lot of money for somebody to buy you can't maintain it so many problems are there ok so there is a real advantage to that penalty being reducing that penalty right having a rate as high as you want ok that's that's needed but what's on the other side if you use too high a rate you'll see later on you will not get very good gains and probability of error ok so what happens if you use too high a rate for instance if you do uncoded storage what happens you cannot recover from any scratch yes you so you can use a cd maybe once twice after that gone can't use it ok nobody will pay you for that also right if you buy a CD or a DVD how many years do you think it should last ok forever ok it won't last forever believe me but but you would at least imagine 10 years right I mean you don't want it to be thrown away before 10 years right you probably want to keep it in safe custody for 10 years at least right so 10 years should should be ok that will mean you need to put enough redundancy there but not too much right so there's a trade-off so in every system that uses error control codes there will be this trade-off and that's a real trade-off in terms of money right it's not some artificial trade-off like I wrote down before in terms of rate and some probability of error that's how we'll quantify it later on but in the real world it's a real trade-off which will make a big difference in money ok that's the first example second example I want to present is should I try the diagram again or no try it again ok so here you go ok you asked for it ok cell phone it's probably very easy it's probably the easiest thing that's out there I think I did a better job drawing this than the CD huh ok good so what does the cell phone do how is the what does it do do you make a call what what's the other most important piece in the cellular system the base station right so you need the base station so ok is that base station enough for you yes that's base station maybe I'll put a hash here ok so that's base station and there's a link here ok so the channel here is a band of radio frequency it's not it's not a wire or anything else it's a band of radio frequencies and and you want to communicate right so there are there are several things to keep in mind in practice for instance most of what we do the cell phone is what this voice right you speak and speech it turns out needs doesn't need very low bit error rates ok well it depends on what compression you but the compression they do in the cell phones requires bit error rate of 10 power minus 3 etc so it means you don't need a very strong code but still there is a code even when you speak your your voice is converted into bits right using some speech coder and then it is encoded with an error control code before it stands for it doesn't go without coding without coding it won't work still you don't turns out you don't need ok but increasingly cell phones are also being used for data right for data you really need a low bit error rate at that time you need very strong coding and strong coding is these days being proposed for the new cellular standards also ok so let's go back to the trade-off now ok let's not worry about the very latest and greatest in cellular communications I'm sure that I enough courses and definitely in this department which is talking about all those things let's talk about the coding trade-off ok suppose I say in the communication from the cell phone to the base station ok there are two parameters right one is probability of error other was transmit power why is transmit power very crucial what do you think it will affect battery life right so it's so so crucial somebody tells you a battery for instance I think even today Sony Sony and Motorola phones have a reputation of having very poor battery time so people won't buy this it's a big problem ok so if you have very if you have no coding if you have no coding if you have no coding at all right your transmit power is going to be really really huge and you know it's not going to work but on the other hand what happens if you put very strong coding very very low rate very very low rate then what happens yeah then that even then there could be some problems ok it's not very clear but you'll see later on it could be some problems you can't just hold on to it for a long time because I think when you speak you you're expecting to hear within this when within the delay becomes a big problem you can't use a very long block length why will block length become large when rate becomes low k by n right the message bit is going to be the same the block length will start becoming very very large and if you put the rate too too low then the block length will become really large it'll take a long time for it to go through and you'll start hearing it'll be like a overseas call you know if you have had some overseas calls cheap calls you'll have to be confused and who's supposed to talk you don't know what's happening that's also not a good experience people won't do that talks ok so you see that this abstract trade-off that we are going to study in the next next for the rest of this course between rate and probability of error ok is a very real trade-off in terms of money in any digital communication system ok but this will be the absolute last time I ever mentioned or ever draw diagrams like this or mention anything close to a real system ok I'll never do that I'll always be talking about rate probability of error only those things I'll be talking about ok you'll have to make that connection later on ok maybe maybe towards the very end I'll give you some examples and more more a better feel for this trade-off but just this motivation for understanding understanding what kind of real trade-offs coding can give you in a system ok any questions or comments at this point ok so if you did not believe error control codes existed out there as I said it's there in each cell phone ok and it's there in every cd driver you buy is every cd player every computer it ships out it's it's everywhere ok it's around all over the place every every system uses error control codes ok all right so that's the that's all the background I wanted to give you let's jump into some some basics of codes ok I thought it's supposed to go to the next page manage to do it ok so so here's a brief look ahead at what the rest of the lecture today and and the next few days will be ok so we'll study what are called linear codes we'll begin with what are called linear codes ok and well there'll be binary as well so let me just make it very specific we will study linear codes and then after that we'll study decoders ok so that's what we will do as we go along ok so I'll introduce one specific type of code ok I haven't told you what a code is ok specific type of an encoder which will do only linear operations ok we'll study that first and we'll try to make some selection out of that we'll not really optimize too much but we'll make some selections for the encoder and then we'll study how to decode it ok it'll be a very simple specific examples and then we'll see in what way in what way is this limited what it can give you what it cannot give you ok and then we'll see how to improve upon it ok so hopefully I think this one week will be enough to by Friday I should be done with all this ok so so it's not it's not it's an old area in coding and we'll go through this really fast ok all right so that's what we'll do in the next few lectures there's one unfortunate part to this course listen we don't have a really good text ok so so you're stuck ok so you either have to attend class take notes and follow up with these lectures online or or try to find some books in the library there are several good books ok and so maybe I should maybe I should write down the names of some books right let me do that so some books that you might want to pick up from some place ok first one I would write down is a book I'll write down the author's name I think that should be enough for you Richard Blahut I believe the book is called algebraic codes for data transmission ok the title is algebraic codes for data transmission that's the book so my my lectures will be based on several books ok I'll write down all these books none of them are available in cheap Indian edition ok if you're rich enough you can buy it from amazon or something paying the dollar rate or or if it's available in cheap edition let me know or it might be in your hostile land right this is one of those solutions might exist ok if they exist then exploit it the second book is by Lynn and Costello use the second edition ok so you need the second edition ok so that's a what's a good book and third book I've heard there's a book by Ron Roth ok this could be available in cheap edition I think I also heard that this book is a good book and I haven't looked at it but I heard the approach is similar to something that I might be doing ok so these are books and there are other books also available out there and if you come across anything I mean bring it to me I'll I'll be I'll be glad to comment on those books ok ok and then there's this grandest greatest books books of them all which is the book by Florence McWilliams and and N. J. A. Sloan ok so I believe this book is like out of print you have to do a special order it's some hundred and thirty dollars it's very expensive but it's a it's a very very good book ok all right so so I'll I'll encourage you to get a hold of one of these books if possible I mean and use that or maybe use some other book also but I would recommend one of these ok so let's jump into linear codes ok so I'll start by describing the encoder for a linear code and what it does and then we'll slowly proceed from there ok we'll see some examples maybe that's all we'll do in this class ok so very simple very simple approach to the whole thing ok so as we said the encoder what does it have to do it has to take k bits in which I'm going to call the message vector m ok m with anything that I put down this will make it a vector ok so in this case m would be what m 0 m 1 dot dot dot till m k minus 1 ok that's the being awake text I don't hear a k minus 1 then I'll have to do something else to wake all of you ok so that's k minus 1 what are each mi is 0 1 right it's just a bit ok so k bits come in this is going to encode I'll simply call it encode now I'll just write down what it does below ok it puts out as I said n bits I'll say the code word c will actually be the message m and then to it it'll there'll be a vector p that's appended ok this is what I'm going to say the encoder is going to do ok my encoder for a linear code should do this I'm going to say ok it should take k bits right I'm going to call them m and that's to put out n bits which I'm going to call the code word c ok so what what do I mean when I write m and p next to each other the kind of concatenated put next to each other right how many bits is m m k bits so what should p be n minus k bits ok so this is the message part of the code word this is the parity part ok and the parity part is n minus k bits all right ok so you see we've made a very nice and simple looking encoder which is at least nice so nice to present ok so what do you do when you actually implement an encoder like this the first k bits can simply transmit as you receive it's exactly the same as the message then after that you have to transmit n minus k other bits how do you do that is the encoding problem right first k bits you know what to do after that for the n minus k bits you have to do this ok for a linear code the encoder ok so let's say the parity p is going to be p 0 p 1 to what p n minus k minus 1 ok so that's my parity ok I'm going to say for the linear code where did I say it's linear each pi is what is the xor of several bits from from what from m I'm going to say ok right so now we've come from 2 power n p 2 power k to some very simple encoding so how many different ways can I do this let me see who's going to be the first to count this I'm going to say you have to transmit the first k bits as it is what you receive and then each parity bit can be an xor of several message bits so you'll have 2 power k into n minus k possibilities ok so you see we have already reduced it by quite an amount ok if you don't believe me you can do some calculations in my lab and convince yourself this 2 power k into n minus k has a very good chance of being much much less than 2 power n c 2 power k or some such number ok so think about why this is a reduction ok at least you can see we're doing very some very small few possibilities for each of these things ok all right ok so that's what the encoder does I'm not going to worry about what the channel is going to do what the decoder is going to do we'll see as we go along but this is what the encoder is going to do ok so a few things again rate is what rate is k by n ok so that's again that definition carries over I'm not going to change that definition here anything else I should mention here it's of importance I don't think anything else is that important ok it's such a simple simple thing ok so what we'll do is we'll we'll describe this in so many different ways to find it to find it to find it and get to a point where we are very happy that we understand the encoder really really well ok so we'll do that very fast and then we'll slowly move on and figure out what we really need to do to understand the decoder you'll see that will take more time it'll require a little bit more effort but the encoder is quite simple enough ok so we'll begin by looking at an example ok so this example will carry over in most of our most of our discussion ok so very simple example but still it captures a lot of lot of what can happen in an error control code ok so I'm going to say in my example I'm going to pick k equals 3 and n equals 6 ok I'm going to send three bits in I'm going to get six bits out so how many message and how many parity bits will be there in the code word 3 and 3 ok three parity bits and three message bits ok so I'm going to say my p0 will be what shall I make it I'll make it m0 plus m1 I'm going to say ok so keep this in mind all my pluses will be modulo 2 whenever I write a plus it will be modulo 2 what is modulo 2 summation it's the same as binary XOR for binary ok so I'm going to just I will I will in the future not write this ok that will be implied in most terms I'll simply write I'll simply write m0 plus m1 ok so in this equation how will you write m0 in terms of p0 and m1 yeah you can write p0 plus m1 ok so you should be very comfortable with that minus 1 is the same as plus 1 modulo 2 XORing twice makes everything 0 ok so that's that's one thing it'll be very very clear in the future when I write there'll be several places where you think there should be a minus but it'll all be plus ok and two times anything will also become 0 ok that's my p0 ok then my p1 is going to be what shall I make it m1 plus m2 ok and my p2 is going to be m4 plus m7 ok good I'm happy ok it's going to be m0 plus m2 ok seems like a very nice and symmetric thing to do I had three bits to put out as parity I just took all possible two combinations and put it out ok so I want you to take so now you remember the first the first discussion right all 2 power k possibilities exist for the message and then only 2 power k out of 2 power n codewords up there right you don't have no only 2 power k codewords are there out of all 2 power n possibilities I want you to make out a table where you have m on one side and m as one column and c as the other column and list it out ok take take two minutes I think it's a very well spent two minutes to write down m is one column and then c next to it ok which is easy m or c m must be only asking must be is there ok so no c I wrote as mp so m0 m1 m2 m I wrote as m0 m1 m2 left to write always ok so what part of c is easy to write m or p again m no so it's very easy simply write 0 0 0 0 0 1 0 1 0 0 1 1 1 0 0 1 1 0 1 1 ok and then p ok which message is the easiest to encode ok what's this one 0 1 1 1 1 0 1 0 1 0 1 ok let me know if I make a mistake 1 0 1 that right ok 1 1 0 0 1 1 0 0 0 am I right did I make a mistake ok so you are happy ok we just made one table but as you wrote it down you could have observed one very important and crucial property for any linear code if my message is all 0 what will my code word be in this way of encoding right it'll be all 0 ok so that's a crucial message for linear linear encoders the way we are seeing a message is all 0 the code would also be all 0 right so you're only doing that one thing ok so I have to tell you what the code is I'm not told you what the code is I've only told given you the encoder the code is the list of all possible code words ok code let me write it here code is set of all code words so remember the code is different from the encoder right you'll see later on that for one code you can have any number of multiple number of encoders you can have more than one encoder ok but the code itself will be unique ok in most cases the code code will be chosen carefully so in our example what is the code ok if we call the code C the example code C is what ok I have to list it out right 0 0 0 0 0 0 0 0 1 0 1 1 dot dot dot you saw it in the column till 1 1 1 0 0 0 so what's the size of my code what's the size of any set the number of elements in it right so the size of the code is what 2 power k in which case it's 8 ok that is the code ok all right so now I have about 10 minutes left so I'm going to try and write the same encoding process in various different ways ok so I'm going to write it in two different ways ok it's the same exact encoding process I did and then we'll generalize from there ok so right now we simply say message and then you make some combinations to get parity it's not clear what this set that we get this the set of code words we get it's not clear what it is ok so we're not able to quickly see any wonderful properties for that set right so we just we're just thinking about it but the it turns out one property we could see was what 0 0 0 0 0 0 0 will be there that's the only thing we could see we couldn't see anything that's more non-trivial maybe maybe some of you can but it's not very immediate we'll see if we interpret it using these other ways of writing it down you see some more wonderful properties will follow ok so the first way of writing down the encoding process in a different way is using this notion of a generator matrix ok ok so what am I given in the encoder if you go back to the previous picture I'm given k and n and then I'm I will have to tell you how I do this XOR what bits of m or XOR to get a particular PA if I do that I completely describe the encoder I'm going to just capture all that information into a matrix form I'm going to say I'll write the code word C ok as m times a matrix G ok I'll use capital letters for matrices without any underlining ok so you don't understand capital letters are matrices typically without any underlining the capital letters can also be sets ok the difference will be clear I hope ok so I won't multiply a vector by a set at least not in this class ok so so for this equation to be true what should what type of a matrix should G be well first it has to be a binary matrix right I mean so the way we did it everything is binary you're doing all mod 2 so it should be a binary matrix and then it should be size k by n right because m is a 1 by k matrix so to get a 1 by n matrix out multiplying with G on the right k should be G should be a k by n matrix so you see immediately G is a k by n matrix ok then is it really possible can I really write C as m times G ok I'll show you how to write for the example and then from there it'll be clear what you would do for the general case ok it's obvious C says it's obvious it is obvious I want to just drive home the point a little bit it's very clear it's very very easy for the example at least it's very very easy ok so for the example I can write C equals m which is what I'll expand m for now m0 m1 m2 then I have to multiply on the right with G right what will I put here for G ok well the first bit of C is what m0 m0 itself ok so you see the first column of G should be 1 0 0 ok so why am I looking suddenly for the first bit of C why did I look at the first column of G ok you should know that much about the basics of linear algebra right when you multiply with the matrix on the right each entry is simply a linear combination of the each column of G right so that's the important thing so you see the first column is simply 1 0 0 what's the second column 0 1 0 what's the third column 0 0 1 likewise now you see you can quickly write even the fourth column right what's the fourth column it has to give you the first parity bit how did I find the first parity bit m0 plus m1 so it should be 1 1 0 ok and what will P1 be I think I did 0 1 1 ok 0 1 1 and then P2 was 1 0 1 ok so if you think about this for a while you will see for any linear encoder any description I give you I can definitely write it in this form you can write C equals m times G ok convince yourself that you can right how do you do it the first first k by k part will be what identity itself and then there will be a k by n minus k part which will be defined by the actual XOR sided column wise ok so now you can go back and look at his answer why did he say 2 power k times n minus k in this matrix what can you pick and what can you not pick the identity is given to you you can only pick the parity part and there are k times n minus k bits there so obviously there are 2 power k times n minus k ways of choosing it so you see just by doing the simple transition to a generator matrix you're able to answer this question like how many encoders will there be in a very easy fashion without having to resort to some well I don't know if he resorted to fancy counting but at least something that looked unobvious initially when he gave the answer okay right when you wrote it down in this matrix form you see immediately there are wonderful results like this are just coming out very easily okay so you see this at least aids in some simple counting you'll see it aids in so many other ways we'll see that as we go along okay so in general this is true any any encoder that I described before can be written in this form as a matrix multiplication on the right the first k by k part will be an identity part the next part will be a parity part okay so this form is called the systematic form for the generator matrix okay so if you have seen this before this form in which you have an identity here is called the systematic form okay okay so I think I'm very close to finishing time so I'll stop here we'll pick up from here in the next class right at this point and then I'll describe the generator matrix and you see so far we've been looking at columns of the generator matrix suddenly if you rewrite this term you'll see the rows of the generator matrix mean more to the code than the columns of the generator matrix okay so what do you think the rows of the generator matrix will be yeah the code was they'll form a basis for your code the code will be a linear subspace all those things you'll see immediately if you look at the row picture in the next class okay thanks we'll see we'll meet again at 11 o'clock tomorrow okay