 Welcome back to our lecture series Math 42-20, abstract out your row one for students at Southern Utah University. As usual, I'll be your professor today, Dr. Andrew Misteldine. Lecture 25 constitutes the very end of our talking about coding theory. I know we're all really sad here. There's so many things we have to talk about, and I just wanted to give us a very short, but awesome introduction to this fun topic of coding theory. So let's summarize what we've learned so far. The efficacy of a linear code really comes down from the balance between linearly independence and linear dependence. So we want to balance these two competing forces. In terms of the column vectors of our matrix H, where H is our matrix that generates the linear code. So for example, to form an NM block code. So if we want to find and form an NM code here, our matrix H must be a P by N matrix. It's got to be a P by N matrix with a nullity of size M. So the nullity determines the size of the code, and thus as a consequence, it determines the message set Z2M. So the nullity is going to go hand in hand with these things. So if we need to encode three bits, we need the nullity of E3. If we need to encode 12 bits, we need the nullity to be 12. Great. Now for the parity check matrices, these are our matrices where H has the form A augment some identity, where we have some control over this matrix A right here. For the parity check matrices, the rank of H here, this is going to be the number N minus M. This will be the size of the augmented identity matrix we see right here. So we need that to be the rank. In particular, this tells us that this number P is going to be N minus M. Great. But still, how does one choose the matrix A? So we've talked about how big the matrix has to be, but how do we choose the matrix A? That's going to be the heart of our discussion in this lecture number 25 here. So the important thing to remember is how large do we set N? How large do we set N? Because after all, this number P is determined by N and M. M will basically be described to us. So how large do we choose N to be? The larger N is, the more room there is for possible code words, thus increasing the minimum distance. We have benefits from separating words far apart from one another. So the bigger N is, the more space we have and we get better error correction. This allows for a larger level of error detection and correction. Thus, a large N allows for better codes, but a large N does not guarantee better detection and correction. That choice comes down to A. A bigger N doesn't automatically have a great error detection. We got to choose A. So that's what I want to talk about right now. How do we choose this matrix A? So remember a little bit of notation. We're going to use the vector EI to denote the vector of all zeros except for a one in the ith position. So this is like the ith column of the identity matrix. Take a arbitrary vector inside of Z2N. So this is the space of encoded. This is the space of code words. That is to say our code is a subspace of Z2N. So given any vector, since it's Z2N, the only coefficients are zero one. So we got to keep track of where are the ones, and that describes our vector. So the vector X can naturally be written as a sum of these EIs. So long as I keeps track of who where the ones are, and we'll call this the index set of our vector right here. So if we multiply X by the matrix H, this one, just by matrix multiplication, we distribute across this linear combination. We see that XH is then a sum of the HEIs. Again, over that same index set. Why is this relevant? Well, one thing to remember with linear algebra is if you take a matrix and you multiply it by the ith column of the identity, this produces the ith column of H itself. So this is a nice little trick from linear algebra. Because after all, if you times by the identity matrix, you'll just get back the matrix H. So if you just take one column of the identity, it will produce one column of H. All right, so we'll come back to that in a second. So HX is a sum of the columns of H, all right? Now let's suppose that X is a code word. The code word for a linear code is the null space of our matrix H. So X should be the null space. That means that when you multiply X by H, this ought to be the zero vector if we in fact have a code word, okay? Now of course, the weight of our code word X is equal to the cardality of that index set. And so if this equals zero, if this equals the zero vector, then the sum of HEI, it's at equaling zero, this is giving the smallest non-trivial combination of column vectors, which combine to be zero. I guess I should say, I forgot to mention that if we choose X to have the minimum non-zero weight of C. So what if we pick the smallest word? After all, the smallest word in a group code determines the minimum distance of the whole code. So what if we choose that smallest positive weight? Then because it's in the null space, this combination gives you zero and because X was chosen with smallest weight, this right here will be the smallest, the absolute smallest combination of columns of H that produce the zero vector. And so whenever you have the columns of H that produce the zero vector, some linear combination, we'd call that the dependence, a dependence relation. Well, since we chose to be smallest, this is a minimum dependence relation. No smaller index set can produce zero better than what we have in front of us right here. So let's call this a minimum dependence relation on the columns of H. And so what we see is the following. The minimum distance of the code is gonna equal the minimum weight of a word, K, because it's a group code. And then like we just observed, this is gonna be the cardinality of this index set. So the size of a minimum dependence relation determines the minimum distance. All right, so our observations can be summarized in our theorem here, 8, 3, 4. Let H be a matrix with Z2 entries or what we call a binary matrix. Let D be the size of the minimum dependence relation on the columns of H. Now there could be more than one dependence relation of minimum size, but we're just talking about the size of that dependence relation with his minimal. Okay, let D be that number. Then the linear code, null of H, can detect D minus one errors and can correct D minus one divided by two many errors. And it doesn't matter whether it's even or odd, so we're gonna take the floor of that number. And the idea here is that this minimum dependence relationship is equal to the minimum distance, like we saw in the previous slide, for which the minimum distance leads to these two observations, error detection, error correction. So that's the idea here. A quick corollary to this theorem, let H be an M by N binary matrix, then the null space of H is a single error detecting code if and only if no columns of H consist entirely of zeros. Now, when I say a zero detecting code, a zero error detecting code, I mean that it can detect a single error. It could potentially detect two, three, 17. By single error detecting, I mean it has at least the ability to detect a single error. Okay, so a single detecting code will happen if and only if no columns of H can consist entirely of zeros. So there's no zero column inside of our matrix. The thing to remember here is if you have the zero matrix, the zero vector inside of a set, this is automatically linearly dependent. Every set of vectors that contains a zero vector will be linearly dependent. And so as you shrink that down, shrink it, if there is any column of zeros in H, then you can take the one size, the singleton set of the zero vector and that would be the minimum dependence relationship. Of course, if you take the empty set, that's actually linearly independent. So you can never get zero for D, but we could get one as a possibility for which means, according to theorem 834, if your minimal dependence relationship has size one, now the only way it's size one is if and only if you have a zero vector, because that's the only way to get a singleton to be linearly dependent. If your size is one, if your D is one, then your D minus one is a zero. And that's basically the argument that's summarized right here. It's a pretty nice argument too. So as long as long as you have no columns of zeros, we can detect at least one error using linear codes. And that's a pretty easy check, right? But just look for zero column of zeros, don't put them in there. Furthermore, the null space of H is a single error correcting code. And again, by single error correcting here, I mean that it can correct at least one error. It could potentially correct two, three, 5,280, whatever, depends on the efficacy of the code, but it can at least correct one. So our code will be a single error correcting code if and only if H does not contain any zero columns. So the first condition satisfied and no two columns are identical. So in linear algebra, how do you get a set of vectors of two vectors to be linearly independent? This thing will be linearly dependent if and only if U basically is not some multiple of CV. Now, technically speaking, we should also say or V is not equal to a multiple of CU. If one of the vectors is a multiple of another, then the sets can be linearly independent because if U equals CV, then the dependence relationship you need is just take negative one times U and add to it CV, which is just equal to U. This would give you the zero vector. And this also includes the possibility that U or V could be the zero vector itself. That's actually why I had to consider both cases right here because you have the zero vector. If U was the zero vector, then yeah, you just take zero times V, you get U. But if V was the zero vector, then no multiple of the zero vector will give us U if it was itself non-zero. So as long as the vectors, if one vector is not a multiple of another, then a set of two vectors will be linearly independent. Now in the special case of mod two, there's only two possible multiples. You either are gonna get zero times V, which is the zero vector, or you're gonna get one times V, which is equal to V. So the only way you could be a multiple of another, I guess what I'm basically saying is the only option here is you're gonna take U in the zero vector or you're gonna take, which honestly that's not a minimal relationship, you would take the zero vector right there, or you're gonna get U and U because the column shows up twice. Let's look at an example of this. So the linear code associated to H1, what'd you see right here? Let's see how good it can do. You'll notice scanning, scanning, scanning. There's no columns of zeros. So zero is not here. So this means we can detect up to one error. That is, we can detect at least one error. Ding! Let's do the next check. Are there any repeated columns? Uh-oh, there are repeated columns right here. So this tells us that the set 101 and the set 101, this is a linearly dependent set and this will be a minimal dependent set. So D equals two. So we get error, so we can detect according to the previous theorem, we can detect up to one error and we can correct up to zero error. So that's not the best, but you know, at least we see that here. So this has an error detection of one. If we look at H two, scanning, scanning, scanning, oh, we have a column of zeros, which means the set of columns, zero, zero, zero, this is a linearly dependent set, which means that our D level is one, which means our detection level is zero, which also means our correction level is zero. So this is essentially a useless code. You wouldn't want to use this matrix. At least for H one, you could detect an error. It has some efficacy to it. How about H three, what do we see here? Scanning, scanning, scanning. I don't see any columns of zeros. And if you look for any duplicates, if you look for like one, one, one, none of the vectors are one, one, one. If you look at one, zero, one, no repeats there. One, zero, zero, no repeats. Zero, one, zero, nope, nope, nope. So everything is, there's no repeats here. So this tells us that our minimal dependence relationship is at least three. I'm not actually telling you what it is, but it's at least three, okay? In which case this tells us that we are able to detect, we can actually detect two errors and we can correct up to one of them. And that's pretty good for the size of this matrix right here. So some things to note about this matrix from our previous discussion. I should mention that this right here is a parity check matrix. So it's the first one. The first one you took A to be one, zero, one, one, zero, one, which is a horrible thing to do. Why repeat the column? This one did a little bit better, right? Here your A matrix is one, one, one, one, zero, one. Basically we made sure not to choose any rows of the identity or any columns of the identity and we made sure not to repeat any. This gives us an error correction of level one. Now it does turn out that this is actually the threshold for this one. Let's make comment here. We can actually argue that the distance here is in fact three. Notice if I take the vectors one, zero, one, and then you take one, zero, zero, and then you take the vector zero, zero, one. You look at that right there. That's gonna be a dependent set necessarily because when you add these things together, if you just take the first vector that's the second vector, that's gonna be the zero vector right there. And so you can see that, oh, okay, I see there that when you look at the second column, you had two ones and a zero, which means that I could write this vector as a sum. You could write this as E one plus E three, which if you put that together one, two, three, then you're going to be able to get a minimal dependence relationship of exactly three. That means this code can accurately detect two errors and correct one and we can't expect anything better than that. And so this also gives us some idea how we're supposed to choose A. We don't want dependence relationship, so we definitely don't want any zero columns or any repeats, okay? But if you want to get a better dependence relationship, what if you want to get a large dependence relationship? It turns out that you do have a bound, a constraint on the number of ones. If you take the column of minimum weight, so this will be of weight one, weight two, excuse me. If you add one to it, that's going to give you the dependence relationship automatically. It might not be the minimal dependence relationship because it could be that some way of combining these guys together goes forward. But if you want a highly effective code, you need to make sure the columns of your matrix from the A side have a lot of ones in it. But you can't have too many ones because then these things become, you can start making combinations there. So we can see that this is sort of a tricky thing to do, but making highly effective codes might be challenging, but to make a one correction level code is actually super easy. Just don't have any repeats in columns of zeros. Basically throw out all the dumb stuff and you're going to be fine.