 In this lesson, we're going to be changing our role around a little bit in this course. Up until now, we've been focusing primarily on learning about different types of ciphers and then using those ciphers to either encrypt or decrypt a message. But today, we're going to be looking at how can we decrypt a secret message without knowing the key? This process is called crypt analysis. The three main ways that we can attack a message by trying to get back to the plain text without having the key. We're going to look at all three of them a little bit today and we'll dig in deeper to some of these in the next couple of lessons. First off, we're going to look at brute force, a method where we try all the possible keys for a given system and see what comes out of them. The second method is using mathematics to try and reverse engineer a key perhaps from having a ciphertext or a plain text available for us to practice with. That's often called a known plain text attack, but often we'll see that we can a little bit later on in the course when a different type of cryptosystem called a public key cryptosystem, we can actually take one part of a key to reverse engineer the other part of the key. But that's a little bit beyond the scope of this lesson, but I promise you will come back to it later on in the course. And then lastly, we're going to look at using statistical analysis. It turns out there's a lot of hidden patterns and text both encrypted and decrypted that we can use to our advantage to make some really good guesses at what the key must have been to create a given ciphertext. Let's start by taking a look at brute force. When we consider the Caesar cipher, recall that there's only 26 valid keys that could be used to make a secret message. So for the given ciphertext we see on the screen, while it's not easily readable, it also wouldn't be too difficult for us to try deciphering it with all the 26 valid keys and just visually inspecting them to see what they look like. And that's what this code is going to do. We can see we set up a for loop that's going to iterate over the range from zero to 26, meaning it'll try all of the numbers from zero, one, two, three, all the way up to 25. Store them temporarily to the variable called test key. And then for each one of those values of test key, it's going to print out to the screen the word key, the value of the test key, and then what the plain text would be using the Caesar cipher when provided the given ciphertext, the current value of the test key. And we set in cipher equal to false, meaning we would like to decipher this message using the function. And when we do that, we can start just taking a look at what we've got. So key zero, P M O L O, I'm not seeing any words there. Same thing with key one, just a bunch of random characters it looks like. Same thing with key two, and we'll keep going. Nothing promising for key three, four or five. And we go a little bit further, key six, nothing. And key seven we can actually see here has some English words. And our human brains are actually pretty good at recognizing English words relative to just a bunch of random characters. So we could spot this out of the list. And lucky for us, there's only 26 items on those lists. It doesn't take very long. We can see that because of modern computing power, the Caesar cipher from back in 300 BC is not really secure anymore. It probably wasn't even very secure for that long back in the day. Let's take a moment and learn about another cipher that's called the affine cipher. This was kind of a first attempt to mix up two different ciphers. So it's a combination of the Caesar cipher and the multiplicative cipher, each of which have pretty small amounts of keys. Caesar 26, multiplicative cipher, assuming 26 character alphabet, only 12 valid multiplicative keys. But what you can see is happening here is that the affine cipher is going to take our plain text value, multiply it by a key, a multiplicative key, and then add to that a valid additive key or a Caesar key. So to see how this would work, here's our plain text alphabet and some numerical correspondences. Let's just say our multiplicative key was three. We'll do that first, just following normal of operations, order of operations. So we can get our first round of cipher text, but we're not quite done. Now we're gonna add on an additive key. We'll just use five for an example here. And you can see that we can get the plain text letter A goes to cipher text letter F, plain text letter B goes to cipher text letter I, and so on. And what's interesting about this mapping from plain text to cipher text is that it would never have been able to be obtained by just Caesar, or just the multiplicative cipher alone, it required this combination of the two. Which means that this Caesar, and combined with the multiplicative, must be creating more potential cipher text to plain text mappings. Let's actually compute how many they would be. So if you look at this algorithm as defined, we would have 12 choices for our multiplicative key. Again, we're assuming just a 26 character alphabet, but if we were to extend this out, we could recalculate these numbers. We've got 26 choices for our additive or our Caesar key. And because those two choices are independent of each other, your choice of multiplicative key has no impact on your choice of additive key. To get the total number of combinations, we can just multiply those numbers together and see that we have 312 valid key pairs for this affine cipher. That's a lot, it's a lot more than multiplicative or individualized, combining them gives us a lot more choices. So is this now secure? Is 312 enough that we could avoid a brute force attack? Let's take a look. Here we've got a similar idea. We've got our cipher text message, and we have two for loops. The first for loop for Ka for our additive cipher, it's gonna try all of the values between zero and 26. But for each of those 26 additive keys, we're now gonna do another for loop where it's gonna try each of the multiplicative keys. So if you look underneath at the output of the code, you can see it's gonna try, first the additive key is zero, and then it's gonna try all of the multiplicative keys, first one, then three, then five, and so on. And you can see none of those seem to be really helping us out. And we'll keep going through all of the additive keys. You have a little selection here. You can see additive keys at 13, and then there's some multiplicative keys of nine, seven, and 11. And then the last couple we would try maybe would be the additive keys of 25. And then just a small selection, a little snippet there, with the multiplicative keys of 21, 23, and 25. So we'd try all 312 of those. We have 312 rows to look for, and it might take a little bit longer to scroll. But believe me, your brain is again really good at finding English out of the gibberish, and you probably already saw this, but it was the additive key of 13, and the multiplicative key of nine, that got us this recognizable phrase four score and seven years ago. So while it might have taken a little bit longer for the computer, maybe only, I don't know, one second, as opposed to 0.1 seconds, our brain still does not take that much longer to parse through all the possible plaintext and pick out the correct one. So sorry, Affine Cipher, 312 value keys, still not enough to avoid a brute force attack. So are there any substitution ciphers that might be avoidable from a brute force attack? Turns out there is. If you think about our random mapping substitution ciphers, so remember, for any substitution cipher, there are 26 factorial different ways that you can map the 26 plaintext characters to 26 ciphertext characters. It's about four times 10 to the 26 valid keys, essentially, if you think about each mapping as a potential key. Now, that's a lot. That is much more than 26 or 12, certainly more than 312. And let's just kind of pretend for a second we wrote some code to do this. I'm not going to show you the exact code, just a little bit beyond what we know how to do right now. But here's our short five letter ciphertext letter, our characters, easy, w, w, a. And let's say we get the computer to actually run through all four times 10 to the 26 valid keys, our mappings. So we can see mapping one, two, and three, we get like r, i, r, d, d, n, that's not a word, w, o, z, z, q, that's not. And then maybe we're like, wow, we got lucky. Mapping number three, jello, that's gotta be the word. But the computer keeps running, it keeps going, and we got some other selections here from later mappings. Turns out that mapping there, three, three, two, four, two, three, two, four. Oh, that's hello. That's another English word I recognize. And so is the one right after that pizza. And later on down the line, there's penny. It looks like all four of those could potentially be the plain text. There's no way for us to know. And it turns out that might make sense, actually, because of every possible of map, we're going to try every single possible mapping of letters. Not just the possible ones for Caesar or affine or multiplicative, which is like this very specific algorithm. But we're literally going to try every possible mapping for a substitution cypher. Then we're going to get essentially every possible plain text word that follows the construction of our cypher text. Which means that we're going to have a lot of equally likely plain text lying around here. And there's no way for us to know which one is which. Setting aside the fact that to try four times 10 to the 26 valid keys, even if you you assume that you can try a million of them per second, would take you 930 times the age of the universe. So I don't think we're going to sit around and do that. This is a substitution cypher that can sustain a brute force attack. Just by trying all the possible ones. There's two things that are going to be working against you. One is it's going to take way too long for it to be feasible. And two, even if it were feasible, you're going to have a lot of potential plain text that you're never going to be able to figure out, which is the correct one. So we're quickly going to see that as our cryptography systems grow in complexity, the idea of using a brute force algorithm to crack those messages is going to become less and less helpful for us. Time and the possible number of plain text are going to pop out that seem reasonable are always going to be battling against us. So we're going to resort to usually more elegant techniques to help us crack a cipher text message. Let's take a look at using some of those mathematical approaches to do just that. So here we have the cipher text from earlier in this lesson that was obtained from the Caesar cipher. And let's suppose we knew just a little bit about this plain text message that that was encrypted to get it this cipher text. This is where the espionage or perhaps some lucky guesses about what the message is about can really go a long way to help you crack the code. So suppose we knew that this phrase in cipher text actually starts with the word if or if in the plain text message. Let's think about how that could help us figure out the key. Remembering our algorithm for the Caesar cipher. We take the plain text value add on the key mod by twenty six or whatever the length of your alphabet is to obtain the corresponding cipher text value numerically. We can convert that back. So taking the fact that we know that the first letter in the plain text is P which has a value of 15 and the first letter in the plain text is I with a value of eight. We could also do the same thing with the second letters M and F for cipher text and plain text correspondingly. And we can set up either one of these equations. We could say that 15s can grow into 8 plus K or in this case and 12 is congruent to 5 plus K both of which yield the same value for K 7 which if you go back in the lesson you actually see was the correct key. So this little bit of information allowed us to reverse engineer our cipher text with the plain text to get the key value. Now Caesar's as you've seen is not a very complicated algorithm. So let's see if this holds true for something slightly more complex the affine cipher that we just learned about. So again here's the same 15 characters that we had from our cipher text earlier in the lesson that were generated from an affine cipher. This is the one that started four score and seven years ago and let's pretend that we were able to make a guess that it was about that message and maybe we thought the first four letters were the word for F O U R. We can now set up the same type of equations using the affine algorithm. So in this case your cipher text letter is congruent to the multiplicative key times the plain text plus the additive key and then this case mod 26. So we can set up these two equations if we were right with our guests about the plain text word for we would know that six is congruent to the multiplicative key times five plus the additive key the six and the five come from the cipher text letters G and the plain text letter F and then the second equation we have nine congruent to the multiplicative key plus 14 plus the additive key and again nine and 14 come from the cipher text letters J and the plain text letter O respectively. So what are we going to do with these two equations these look a little bit harder to solve they're not just nice linear equations that we could subtract things from both sides we actually have a system of equations with two unknowns that we have to figure out how to solve. So let's take those two equations and set them up as a system just kind of rearrange them and tidy them up but the same two equations are here on the screen and we're going to use the what's known as the elimination method to solve for just one of these variables first and then use that information to solve for the other and by stacking these equations this way the first thing I notice is that we have these additive keys being added to both of the equations right there at the end and kind of similar to what we've seen with the Euclidean algorithm I'm starting to think maybe I could like subtract off one of these from the other and in fact let's let's do just that let's subtract the second equation from the first and when we do that we get nine minus six which is three on the left-hand side of the congruence 14 km minus 5 km gives us 9 km and the k a minus the k a they subtract out to zero or think about that is canceling out and now we're left with this congruent statement that has only one variable left to solve you know that three is congruent to nine times the multiplicative key so to solve for that multiplicative key remember we can't divide by nine or working in these modular arithmetic systems because we might end up getting a non integer which is not really helpful for us when we're trying to deal with numerical values that correspond the numbers and letters so let's instead of dividing by nine to solve for the multiplicative key let's multiply by the inverse of nine and in mod 26 remember the inverse of nine is just three so on the left-hand side we really get three times three and on the right-hand side we've got nine inverse times nine those two are multiplicative inverses remember the whole point of that is that just product is one so we're left with one km on the right and we can carry out the multiplication on the left to get our first part of our affine key we now know the multiplicative key used was nine and if you go back and rewind a little bit in the lesson you can verify yes that is in fact the correct multiplicative key but let's try and use that now let's go back to that first equation and let's plug that value in or evaluate this congruency when when the multiplicative key is nine to get this congruency statement nine is congruent to 14 times nine plus the additive key which means that nines congruent to 126 plus the additive key which means that negative 111 is congruent to the additive key remember we're talking about mod 26 so we can take that negative 111 mod by 26 and find out the additive key is congruent to 13 and again if you were around just a few moments ago you know that is the correct additive key for that ciphertext provided we worked with an affine system so we were able to reverse engineer that little bit of known plaintext pair it up with the ciphertext to get these two parts of the key that we needed and now that we know these two parts we can undo the rest of the message so there's a lot of power and just getting one or in this case two letters correct you can use that to get everything you need to unlock the keys and get the rest of your message in the last part of crypt analysis that we're going to introduce today is called statistical analysis and this one has been going on for a long time this idea of statistical analysis really starts with the idea of letter frequency which a lot of you I think you've probably already picked up in this course is that some English language letters show up a lot more than others which means that our ciphertext are going to have some letters that show up a lot more than others when we do those one-to-one mappings that we've been looking at so far and this idea has really been around since about 800 AD by this era of philosopher Abu al-Kindi who's been credited really for the first person to formalize this idea of looking at character frequency and counting it up and then using that to break ciphers of the day. So as I mentioned some of you might have already picked up on this or maybe you've heard about how the letter E is the most commonly used English language letter in fact the letter E makes up about 12.7% of all written characters in English whereas letter M makes up only like 2.4 so the full the full chart is over there on the right hand side of the screen will be using these frequencies in this course to help us make some good guesses about what are some plausible mappings of plain text to cypher text since we know we can't try all of the mappings we're going to be using some statistics and probability to help us narrow down the set of all of the possible mappings to just the most likely ones and single letter frequencies are a great way to start but we'll see that we can use two letter frequencies also known as diagram frequencies to do an even better job and we can continue to refine our statistical analysis so that we can really do a great job about figuring out what is the most likely key and that would lead us to the most likely plain text using all the patterns and statistics of the English language it's really powerful more than we can fit into this lesson so that's what we're going to start covering in the next sequence of lessons is how can we unlock the secrets of the English language combined with the power of Python to help us start cracking some more complicated cyphers