 This talk will be about the following problem. Does P equal NP? Here, this is a problem from computer science and P means the set of polynomial time algorithms and NP means the set of non-deterministic polynomial time algorithms. So I'll be giving an introductory talk on this where I'll first explain in more detail what P and NP actually are and give a little bit of a history and then explain the arguments for and against this question. So let me start with an example. Suppose you take the number 61669 and I ask the following question, does it factor in an untrivial way? So obviously I don't want to factor one time 61669. And if I asked you for the answer and made you do it by paper and pencil, you could do it. You would just check two, three, five, seven and so on as factors and it would be an extremely tedious calculation that you could do it. But what you notice is that this calculation, it's pretty slow. On the other hand, if I ask you for a different question, what is 743 x 83 and you had to do it by pencil and paper, that would be easy because multiplication is fast. And you would find the answer is 61669, which is the number we first asked about. So what we see is that it's pretty difficult to find this factorization 743 x 83. But once you've found it, it's very easy to check that it is a factorization. And now P stands for algorithms that are fast, roughly speaking. So this is not actually correct, so I'll put this in inverted commons. So that means it is fast to find a solution to a problem. So for example, multiplication is fast. On the other hand, NP, we will see that it is fast to check that a solution, a solution once you've found one. So the difference between P and NP is the difference between finding a solution and checking that there's a solution that somebody else has come up with is in fact correct. Well, the problem here is I've used this word fast. And the question is, what does fast mean for algorithms? And we want a sort of general definition that doesn't depend on what sort of computer you are using. It doesn't depend on whether you're using a pencil and paper or a computer that can do a billion operations a second. So what's the definition of fast? Well, there's no very precise definition of fast, but there's a reasonable substitute for it, which is due to Cobham who came up with this in the mid-60s, where it says that fast for algorithms is roughly equivalent to polynomial time. So let me explain what this means. Well, polynomial time means the time taken for an input of length n should be less than or equal to some polynomial in n such as a n to the a for some constant a. So a is some constant and n is the length of the input and the time should be at most n squared or n cubed or something. You notice that the n is the length of the input. So if our input is say 743 times 83, then n is equal to 123456, not 743. And similarly, if we are factorizing 61669, we don't want to factorize it in time that's polynomial in 61669, which should be really easy, but we want to factorize it as a polynomial in the number of digits, which is 5, which is very much harder to do. So fast algorithms roughly correspond to polynomial time algorithms, but you can find plenty of counter examples. For example, the simplex algorithm in linear optimization is a pretty fast algorithm in practice. In theory, it's not polynomial time because there are some very, very rare cases when it takes a very, very long time. But these cases are so rare that you can practice it's a fast algorithm, although it's not polynomial time. On the other hand, you can also come up with polynomial time algorithms that are not fast. If your algorithm takes n to the power of 100 steps, then that's polynomial time. But if n is 10, this time will be so long that it's absolutely useless. So for example, if we do multiplication, this is polynomial time, because if we take two numbers, if we take an n-digit number times an n-digit number, the obvious algorithm that people may or may not still learn in high school takes about n squared steps. Well, it's difficult to say exactly how many steps it takes, so we say it takes o n squared steps, where o means it's most of constant times n squared, and we can't be bothered to work out what this constant is. Actually, slightly surprisingly for such an easy and well-known algorithm, this is not the best algorithm for multiplying two numbers in general, there's a faster algorithm using the fast Fourier transform that takes about o n to the 1 plus epsilon steps, where epsilon is as small as you like. All these numbers like n squared and n to the 1 plus epsilon shouldn't be taken too seriously, because the actual amount of time your algorithm takes depends on all sorts of things like details of the hardware of your computer and so on. There are always extra factors of powers of log n and so on that people don't really bother putting in here, so you should take all these expressions with a bit of a grain of salt. So, polynomial time is usually denoted by p, so p is polynomial time, meaning you can solve the algorithm in polynomial time. Np stands for nondeterministic, which they've chosen a shorter word, polynomial time. So what does this mean? Well, this means you're allowed to make an inspired guess if you check it. So, for example, a nondeterministic algorithm for factorising 6, 1, 6, 6, 9 might go like this. If you look at your daily horoscope in the newspaper and your horoscope says, well, your lucky number is 83, so you divide 6, 1, 6, 6, 9 by 83 and you find the answer is 743. And you can do this division in polynomial time and that checks that the answer is correct, which is probably a good idea because I don't think astrology columns are really a particularly reliable source of information. So, the main question is, the question is p equal to np? So, if you can solve a problem in nondeterministic polynomial time by making inspired guesses, is there an honest way of solving it in polynomial time without needing inspired guesses? So, I'll just very give the history very briefly. The first person who asked such a question may have been Gordell in about the 1950s. He wrote a letter where he sort of asked, suppose you've got a mathematical theorem of length, mathematical theorem, can you test whether it's got a proof of length n in a time that's linear or quadratic in n or something like that? So, he was asking, can you find a proof of length n in polynomial time? And he didn't really push this much further. And the p equals np question is usually really dated to an article written by Stephen Cook in about 19, in the early 1970s, maybe 1971. And the notion of np completeness and the p equals np question was soon after expanded by Karp. So, in particular, they introduced the notion of being np complete. So, an np complete problem is a maximally difficult, it's a maximally hard np problem. What this means is that if you can solve it, and you can solve all others by solving polynomial time. So, the question is, so in order to answer whether np is equal to p, all you've got to do is to show that one np complete problem can be solved in polynomial time. So, let's see some examples of this. So, first of all, let's ask, is n composite? Well, this is certainly an undeterministic polynomial problem, because we've just seen if it's composite you can make a guess for the answer and check it's correct. Then we can ask, is it in p? Can we do it in polynomial time without guessing? And the answer is yes. And this was first proved by Agra, Val, Kaila and Saxona, who found an algorithm that would test whether a number was prime or composite and they were able to prove that algorithm could do it in polynomial time. Incidentally, the algorithm probably wasn't the first algorithm that would test numbers in polynomial time. It was the first algorithm which they could prove that it did it in polynomial time. There were some other known algorithms that probably factor in polynomial time, but the trouble is proving they factor in polynomial time requires using some unproved hypotheses such as the generalized Riemann hypothesis. So, telling whether or not a problem is in polynomial time can be really tricky and difficult to decide. Another example is, well, we first asked, is n composite? We can ask the second question, find a factorization of n. And again, this is an undeterministic polynomial question and we can ask, is it in p? Can we find a factorization in polynomial time? And this is a completely open question. It's one of the fundamental questions in computational number theory. And I'm not even sure which way I would guess for the answer. On the one hand, people have searched really hard for fast factorization algorithms and never found a polynomial time one. On the other hand, the algorithms are getting better and better and there are all sorts of really deep ideas that we haven't yet used. So I have no idea whether the answer to this is yes or no. Notice, by the way, that the algorithms to test whether n is composite, the fast algorithms don't do it by actually finding a factorization of n. They do it by rather more subtle means. Go and see a course on number theory if you want to know how. So now I want to give an example of an NP complete problem. The classical example of this is the traveling salesman problem. Salesperson or whatever. So this problem says suppose you've got various towns and a salesman wants to visit all these towns in the shortest possible way. So we ask the following question, is there a path going through all the towns of length less than or equal to k? So you might sort of guess that a path is going to go around like this. So this might be a reasonable candidate for the shortest path and this is obviously non-deterministic polynomial because if there's a path of length less than or equal to k, then you can just write down the path and check it's got length at most k and you can sort of, that basically takes linear time because you just have to add up the length. On the other hand, finding the path seems to be extremely difficult. So this is an example of an NP complete problem. If we could solve the traveling salesman problem in polynomial time, we'd be able to solve all other non-deterministic problems in polynomial time. So all you have to do to show that p is equal to NP is to find an algorithm for solving the traveling salesman problem in polynomial time. And so far nobody has managed to do this. So we have the following question, is p equal to NP? And there is no easy proof either way. There's no easy proof and no easy disproof. The first result suggesting that there was no easy way to do this was there's a result by Baker, Gill and Solovey. And what they did was instead of looking at algorithms, you look at algorithms with oracles. So an algorithm with an oracle means that your algorithm for doing something is allowed to stop every now and then and consult a wizard who knows the answer to a large collection of problems. And so they will stop and ask the wizard some question and then take the wizard's answer into account and carry on. And the wizard doesn't know the answer to actually every question. The wizard or oracle only knows the answer to a certain collection of problems. And what Baker, Gill and Solovey did was they found a special sort of oracle such that if you look at the polynomial time algorithms with oracle x, this is equal to the non-deterministic polynomial times with oracle x. And similarly they found an oracle y such that p of y is not equal to NP of y. So if you allow algorithms to have an oracle, then this question can be either true or false. And this is a bit disturbing. The reason being that most of the methods we know for proving theorems about speed of algorithms go through with very little change to algorithms with oracles. This means that if there was any proof using easy techniques of this question here, then it would also apply to algorithms with oracles. But that actually depends on which oracle you have. So the proof can't work for all oracles. So now I want to say a bit about whether... Let's look at the evidence for or against the question of whether p is equal to NP. So the first thing you do, if you've got an unknown question, or a subject you don't know much about, is to ask about expert opinion. So let's consult this. Well, the expert opinion for p not equal to NP, about people have done surveys and something like 90% of people working on it think the answer is that they're not equal, and for p equals NP about 10% think the answer is yes. Well, you've always got to be a little bit careful about expert opinion. If you wanted to know about some results and say parapsychology or astrology or something and asked for expert opinion, it might not be terribly reliable. The trouble is that any expert on parapsychology is probably chosen to work in parapsychology, which means they probably haven't yet figured out that parapsychology is complete bunk. So expert opinion is fine as long as your subject is non-flaky. Well, that's okay because computer science and computation is definitely a non-flaky subject. So expert opinion in this case is probably reasonably reliable, and the default assumption seems to be that p, we should sort of expect that p is not equal to NP, but there's a certain amount of reasonable doubt about it. So now let's, for another argument, let's observe that it's easy to prove upper bounds for the time to solve something. On the other hand, it's hard to prove lower bounds. And the reason for this is very simple. If we want to prove an upper bound, all we have to do is to check one algorithm. And we just have to show that this one algorithm is fast, and that gives us an upper bound for the time taken to solve a problem. However, if we want to prove a lower bound, we need to check all algorithms. Of course, we don't need to check them individually one by one, but we somehow need some argument that would check all algorithms because we've got to check that every single algorithm doesn't somehow cleverly solve our problem. So, upper bounds you only have to check one thing, but lower bounds you have to check an infinite number of things, so it's much harder to do this. This means if we are unable to decide how fast the best algorithm is, it's more likely to be quite slow because it's the lower bound that's hard to prove. So this suggests p is not equal to NP. Because if p was equal to NP, this would mean we have an upper bound. Showing p equals NP means we need to find a polynomial upper bound for an NP complete problem, and we haven't managed to do that, which sort of hints that maybe there isn't one. Well, let's have a counter argument. Well, the counter argument is that it's not always easy to prove polynomial upper bounds. So it can be very hard to find polynomial algorithms. So let's have an example of this. Suppose you take a planar map and you want to colour it with four colours. Now, Apple and Harkin showed that there is a polynomial time algorithm for doing this. The trouble is it's incredibly difficult to prove that there is a polynomial time algorithm for doing this. In fact, as far as I know, no human has ever actually read a proof of this. Okay, so how do you prove it if no human has read a proof? Well, the answer is a lot of the proof was done by computer calculation. You have to check enormous numbers of complicated cases to show that the algorithm they gave actually works. So it can be, you know, there may be polynomial time algorithms that are really difficult to find and prove. So the conclusion of this is that algorithms are really very poorly understood. So although this suggests that P is not equal to NP, what it's really doing is it suggests that either P is not equal to NP or that we're kind of stupid and don't really understand algorithms. And as we're kind of stupid and don't understand algorithms, it doesn't really give a whole lot of information. So there's some very weak evidence in favour of the fact that P is not equal to NP. Now I'll come on to something that I think is somewhat stronger evidence. This depends on the following principle, slightly informal principle, which says the only way to tell what a programme does is to run it. Well, that's for general programmes. I mean, most of the programmes you actually come across are specially written to make them easy to understand. I mean, they have to be documented and written in a simple way and so on. But these are very untypical of most algorithms. Most algorithms are almost impossible to understand. If you've ever tried to read someone's poorly written undocumented code, you will know what I mean. It can be really hard to figure out what code is supposed to be doing. In fact, there's a competition called the obfuscated C contest where people deliberately try and write code in C. That's as difficult to read as possible. I don't really know why because code in C is always a bit difficult to read if you're not trying to obfuscate it. And there are some really impressive examples. For instance, here is an example written, I think, by Brian Westley. And so, I have a problem with you. What does this programme do? And even if you're an expert experienced C programmer, if you look at this, you probably haven't got the faintest idea what the programme does and it would take you several hours to figure out what it does, even though the main part of it seems to be only two lines long and seems to be little more than a couple of definitions. What it actually does is it calculates pi by calculating its own area and then dividing that by its own diameter. You see that the programme sort of looks a bit like a circle. So that's just an example to show how difficult it can be to figure out what a programme does. Incidentally, that's one of the easiest programmes in the obfuscated C competition to understand. If you go and look at some of the others, they are absolutely bizarre. I mean, you sort of look at them and there's just no way you can figure out what an earth is going on there. There's an extraordinary amount of ingenuity and time wasting going into this competition. So the point is that this is that examining the code for a programme or an algorithm in general gives no useful information beyond what you can find by simulating the programme or running it. Another example of this is the 3x plus 1 problem. So this is a rather famous problem. Because you take an integer x and if it's even, you map it to x over 2 and if it's odd, you map it to 3x plus 1. And you keep iterating this and you can ask, do you end up at 1 for all integers? So this is a very simple programme. It's all hard to think of anything simpler than doing this to an integer. You just ask, you know, if you iterate that often enough, do you always end up at 1? And nobody knows. This programme is incredibly difficult to understand its behaviour. So even for very simple, clear programmes like that, it can be very hard to understand what they're doing. So let's have an example of an NP complete problem. Well, I gave an example earlier of the travelling salesman problem, but it's a little bit difficult to see that it's NP complete. It's obviously NP, but completeness is a bit difficult. So here's a problem that's almost obviously NP complete. So it's going to take as input a programme x of length n written out and it could be a Turing machine of length n or something. And you have to ask the following question. Is there some data on which the programme halts? In less than a way you could do n steps. So you fix a number n and you want to have an algorithm that takes most polynomial time in n, which will tell you for any programme of length n whether there's some data on which it halts. And this is obviously NP, because if we're given a programme and it halts on some data of length n, then we can just write down this data by randomly guessing it and check by simulating the programme that halts in most n steps. So it's obviously NP, we just simulate our programme. It's a little bit more difficult to check that it's NP complete. And the point is that since we're allowed an arbitrary programme here, we can just choose the programme x to simulate any algorithm. And from this, if you can answer this question in polynomial time, then by using x to simulate any algorithm, you can basically check whether any algorithm halts on some input in polynomial time. So in some sense, this problem here is the archetypal example of an NP complete problem. Well, suppose we try and answer this question deterministically. So we can see whether some data on which the programme halts by guessing the data. So here's a deterministic solution. What we do is we run x on all possible data of length n. And we check for each of these to see if x stops. So we can certainly check to see if there's some data on which x stops at length at most n. Deterministically. And you notice we only have to run it from data of length n because if we're running the programme, we want to check to see if it stops at most n steps. So it can't read more than n data points. And how long does this take? Well, it takes about 2 to the n steps because there are 2 to the n possible choices of the initial data of length n, assuming the data is in binary or something. And we notice the problem. This is not polynomial in n. So the obvious way of solving this problem doesn't run in polynomial time. Well, what can we do in polynomial time? Suppose we use only n to the a steps, which is some polynomial. Then we can check all inputs of length less than or equal to about a times log of n, which is certainly far less than the inputs of length n, which our programme might use. And so what we want to do is find a polynomial way of checking all inputs of length n. And as we've seen, the obvious polynomial time algorithm only checks inputs of length at most a times log of n. So the problem is can we do it faster? And in order to do this, we need to understand all programmes of length less than or equal to n. And we need to understand them well enough to see where they stop in inputs of length n. And this just seems really unlikely because it sort of violates this principle that you can't tell what a general programme does except by simulating it. And here we're asking a question about all programmes and this strongly suggests we can't tell what they do except by simulating them on all possible input data. And we don't seem to have time to do this. So this sort of suggests that p is not equal to np. But of course this argument isn't watertight. I mean I've been waving my hands vigorously at various points in it. So what you can do is you can say that p not equal to np is a sort of quantitative version of the principle that we cannot understand programmes except by running them. So roughly speaking, if p was equal to np then we would somehow get some understanding of all programmes on input data. And we would have to do that in a way that was faster than just by running them all on the input data. Conversely, if we could understand programmes without actually running them then this might give us a way to prove that p was equal to np. So this seems to me to be quite strong evidence in favour that p is not equal to np because it's just encoding the well-known fact that it's really hard to understand what a general algorithm does but there's no easy way to make that argument watertight. So let's come to argument number four. Suppose p was equal to np. Well, this would imply that we can check for a proof of length, less than or equal to n of some theorem in polynomial time and polynomial time is sort of usually fast. Well, this has a rather unfortunate consequence. The consequence is that mathematicians would all be unemployed because our job is to find proofs of theorems and if there's a fast way of doing this just by getting your laptop or something to run an algorithm in polynomial time then we'd all lose our jobs. So this provides... Well, maybe it's not exactly an argument for why p is not equal to np but it may be an argument for why we all hope that p is not equal to np because otherwise we'd all get fired. As a counter-argument, mathematicians seem good at finding really long proofs. The most spectacular example of this is the classification of finite simple groups and the proof of this is about 20,000 pages long. I mean, nobody really knows exactly. The proof is such a spread out so much it's rather hard to tell. So this kind of suggests that in ideal cases, mathematicians really are rather good at finding proofs in polynomial time. I mean, we certainly didn't take exponential time to find this proof. So it's not clear what's going on. This sort of suggests that maybe proofs of nice theorems can be found in polynomial time or at least found fast. So, you know, mathematics has a lot of nice theorems where there's a lot of structure and lots of horrible questions like this 3x plus 1 problem where there seems to be very little structure to get hold of and maybe there is, you know, nice theorems we can find proofs of in polynomial times and nasty theorems we can't. I don't really know. So, argument 4 is sort of really unclear. I would say this slightly suggests that P is not equal to NP but the fact that we can find long proofs means we should be a little bit hesitant about taking this argument too strongly. So, a fifth argument, which is actually similar to argument number 4 is that if P is equal to NP, this implies that most internet encryption can be broken. Well, maybe not because polynomial time might be really long. So, if a polynomial time method for breaking this was, say, end to the 100 or something, then this wouldn't actually be much use for breaking encryption. But an awful lot of the things like the RSA algorithm and most other encryption algorithms used on the internet could be broken if we had a fast way of solving non-deterministic polynomial problems. Well, I haven't heard of anybody breaking all encryption on the internet but it's not quite clear what this proves. I mean, if someone had figured out how to break encryption on the internet, they'd probably just keep quiet about it and make a huge amount of money. I mean, or if some national intelligence agency had somehow managed to prove that P equals NP, it would sit there quietly reading everybody else's secret communications and would be very careful not to tell us about it. So, the fact that P equals NP implies most internet encryption can be broken and actually provide much evidence for P not being equal to NP. So, my conclusion is that my confidence that P is not equal to NP, I'd say about 90% and P equals NP about 10%, meaning I'd be willing to bet 10 to 1 that P is not equal to NP. This 10% is quite high basically because my natural instinct is to put this much lower but the problem is that humans have a very strong tendency to underestimate the chance of unlikely events. So, when people say they're 99% certain of something, they're wrong far more than 1% of the time. So, I'm going to be a bit cautious and give these odds for P equals NP.