 Okay, I want to start with a better answer to a request. I don't think I justify the answer quite well. So the request was for solutions for PAT to provide binaries rather than just the handout. Well, I'd love to do it, but I cannot do it for two reasons. One of them is that there are no such thing as a binaries for Python because there is bytecode, but you can reverse engineer it in a few minutes. So that would leak the solution out and the project took quite a while to prepare and we cannot change it every year, so it must not leak out, hence you are getting only hard copies. And there is an educational benefit to getting just the hard copy and it is that you actually need to read what the solution is. Well, you could just retype it, but I don't think it will work that way. So that's how it is, I'm sorry. Yes, you could, but then you couldn't test it on your own machine, right? Because you would need to always go through the autograder. The binaries would need to reside on the server, so you would need to somehow check in. It would need to be shipped to the server. And so that's something I will insist on. I'm sorry. If you did not finish the assignment, we expect you at the very least to understand what the right solution is and fix the few bugs by taking the hard copy of the solution and putting them in. Test cases on PA1, we are going to release, push into your repositories the test cases. Not all of them, but I think the strategy that we'll use that will push back five test cases for everybody based on which are the most frequent ones that people failed. So if you failed 10 test cases, you'll get five back those that caused most problems for everybody. I think they should. Yeah, I don't know now. Okay, that was probably a problem. The question was, what do we do? Can you ask for a resubmit after you picked up the... I see. Oh, you still can pick them up, right, absolutely. So you can just come to the office hours tomorrow and pick them up. Yeah, there is a stag on Sean's and T-boss's desk and you can pick them up. Excuse me? I don't know if they are going to be there, probably not. I don't think either of them has office hours after class. But tomorrow you can pick them up. Okay, so today, I just want to say that we are going to give extra credit for any bugs that you find in the handout, solutions, starter kits, and we'll keep track of it on Piazza. Today, Thursday is no laptop days and that's good because there is a lot of material that we want to cover that you may want to think about every slide that we'll go about. So parsing, why do we care about parsing? You want to make sense out of these sentences? Sentence like this one. See a problem that shows up in parsing of the sentence? So depending on whether there is a serial comma or not, depending on whether you put a comma or not, the association of Mother Teresa and Pope could be, well, they could be my parents. That was the meaning as written because there was no comma. Or they could be the people to whom this lecture is dedicated. And you can guess the meaning from the context, but without the comma, indeed the meaning was that they are my parents. And the second one is here. And again, there is an error. There is a hyphen which changes the meaning to, instead of seven doctors, these are doctors who are seven feet tall or wide, or I don't know. That's one of the two. And then a meaning of this, which is more relevant to the class, can you see what the ambiguity might be here? Right. So to put it more concisely, the answer is good, but to say we are really asking, is this else associated with this if or with that if? Right? It could be one or the other. And so in general, parsing is the process of taking text and turning it into some sort of structure, usually a tree that shows how things are associated, how they're evaluated. It's true for natural language. It's true for computer programs. And there is a lot of text out there, just a thing of HTML, JSON, and all the things that you will need to parse, in addition to programs. Grammar is other things that describe strings. Essentially a grammar is what the description of a set of strings, that set of strings that the grammar describes is a language. And we have seen it before. But let's recap it on the case of regular expressions. So regular expressions are described with a grammar that is inductive, recursive, almost like any other grammar. And what it says that there is a base case and any character C is a regular expression. So A is a regular expression, and so is B, and so is C, and so on. And then the inductive case, if we find regular expressions even in E2, then the alternation, the concatenation, clean closure, and the regular expression in parentheses, these are regular expressions as well. So the grammar therefore describes which strings are sort of legal strings, legal program strings that we want, and indirectly describes which are not. So give me a few examples from the language. Just one quick example that is derivable with this grammar. A will work. One more. So we'll do this. This is from the language clearly. Give me a few that are not from this language. So A dot, well, now it really depends whether dot has a special meaning or not. According to our description, the dot would be probably just a character, even though in the regular expression used in programming, dot does have a special meaning. So let's do something unambiguous here and say maybe give me something that uses what special characters we have in this definition. It is just the bar and the star and parentheses are special. The rest are assumed to be normal characters. Two bars next to each other should be illegal, right? Maybe A separated by two bars or star A would all be illegal characters, okay? So hopefully this is clear. Now we use this notation to describe grammars. Again, it's something that you have seen, but it's good to give names to the notation. So what are the red things? The red things are the characters that you can actually see on the input in those strings. These are called terminals because the grammars used to derive the string and you derive it using what are called non-terminals. These are just symbols that you need to rewrite all the way down to terminals. Non-terminals cannot appear on the input. They need to be rewritten to terminals and terminals are characters because that's where the rewriting the derivation terminates. So terminals are the characters, non-terminals are things that are recursively defined. There is a star non-terminal, which is the one that is mentioned here on the left. It's the one from where we start deriving the strings. And productions are five in this grammar. One, two, three, four and five. We list them more compactly this way, okay? You may notice something interesting in the grammar and that there are two vertical bars, a red one and a blue one. Are they really the same bars? One of them refers to the input character and the other one is essentially a special character for the grammar. It separates the productions from each other. So how would we typically distinguish between them? Usually our editor does not have colors like my PowerPoint does. We would escape them and perhaps we would use quotes around here or backslashes, whatever the particular tool escapes. But now you could understand that even though the grammar is just a bunch of characters, some of them refer to special symbols in the grammar like this bar or this, and some of them really refer to input, to terminals. Questions about this as we go on, okay? Can you show a grammar for a grammar? You could show a grammar for a grammar. Grammar is a sequence. Grammar over grammar would be what? It starts with a non-terminal, so something like ID, okay? Then it has what? This symbol, right? Then it has perhaps IDs intermixed with special characters. Let's see if I can do this concisely. Maybe you say it's an ID or let's say character and you need to have one or more of them. Actually, I erased too much, but yes. So ID... So this one here is this, okay? We'll put this in quotes to denote this itself. And now we can mix, say, ID and characters. So this would be the non-terminals, characters, one or more of them, and then the whole thing can repeat, can have zero or more of them because you have multiple such productions. This is not quite complete. It doesn't allow these shortened productions here, but you get the idea. Okay, so here are the grammars. It's a grammar, not a grammar. Not that I really care, but it's sort of fun to realize that not all writing is due to bad grammar, which is probably what people tell you. So this mistake itself is not a grammatic mistake. It's a lexical mistake. It's a mistake that happens sort of in the word. You could say in the terminal rather than in how terminals are put together. So if this was a programming language, this mistake is a lexical mistake, a mistake found in the lexical, in the very front of the compilation interpretation, not in the parser. And you should start learning where mistakes in programs are found by the compiler interpreter. So this one here is a lexical mistake. So grammars and languages. I want you to tell me a grammar for this language. Language of one B followed by one or more As. So this is a language that contains this string and then that one and that one and so on. Write down a grammar and tell me, I want to see whether we end up with just one grammar. It does not contain an empty string, right? So it contains exactly all strings of the form B and then followed by one or more As. This is an exercise that is in fact too simple for an exam. So you should be able to do this quickly. Clearly the definition will be recursive because we need infinitely many strings and the only way to define them would be through recursion. Okay, so somebody wants to suggest the grammar. Let's do S since it's usually the symbol I used R only because it was the grammar for regular expression. So S, okay, so do we need, so this is, that's not what you said. You need to speak a bit louder. Okay, that should work. One more. So does this generate the right string? Well, the base case is, okay, how about this? Now what we can generate is BA, right? When we go directly to the base case, when we go through recursion, we generate SA and then the S is rewritten recursively into BA and A and so we seem to be good, okay? An alternative grammar. Can somebody suggest another one? One that will differ in significant ways, okay? Yes. So S would go to B and E would be a sequence of one A or that should work too, right? Okay, so what we see is that we have one language, one set of strings, and many different ways how to describe it. Do you see any differences in these two grammars? Clearly we pick different names. In one case we use just two productions, one and another one. Here we used three, one, two and three productions. The productions are the rules from the left-hand side of the non-terminal to what it is rewritten to. But I wonder if you can spot some significant differences that might make a difference. Excellent. So the answer is that here the S is on the left. So S is the recursive symbol and it is on the left of that right-hand side and then here E is on the right-hand side, okay? So this grammar here, I'll erase it actually and we can look at one here and another one there. And there is a left recursive grammar, which is this one. It's left recursive because S is right here and this one is right recursive because the recursive symbol A is on the very right of the rule. It may seem insignificant, but we'll see in a few slides that it makes tremendous difference in some parsers. Some parsers cannot handle left recursive because they would get into infinite recursion. Could you give me a recursive grammar that is neither left or right recursive? Well, so if I change this to that, that would make it left recursive because now the non-terminal A is on the left. See, this one is not recursive because this A just goes here and there is no loop going back to itself. So not entirely, okay? So it's neither left or right. Okay, let's try, please. Oh, I see. We've seen it in lecture two, I believe. This is empty string. Good question. So this is an empty string, okay? We could do BA+, but if we do BA+, that's really just an acronym, a shortcut for saying... So BA+, is essentially something like S goes to BA and A is either 1A or AA. So if you expand the plus, you end up with a recursion also. And you could write it as a left recursion or right recursion. So let me tell you what it means to be neither left nor right recursive. It means that the symbol cannot be all the way on the left or all the way on the right. There must be some terminals in front of it. So how about if we do A is... It has now two As, one on each side. So it generates an even number of As. And we could use it to, of course, define even our language. All we need to say is, well, let me call this A', to distinguish it. It could be S is BA', or BA, or this should be epsilon also. And I think that's it, right? So now it can generate both odd and even number of As following B. And it is neither right or left recursive. But let's start doing something more fun. So the parsers that cannot handle left recursion are quite common, so it's good to know what the left recursion means. And we'll soon see why that's the case. But we can rewrite the grammar to use a right recursion. For example, this, you can rewrite into that. That's not surprising. This is what we've just done. The more interesting case is what you do with a grammar of expressions like this. This takes a while, so I don't expect you to come up with the answer right away. But maybe you have the idea of what we could do to remove the left recursion from such a grammar. You will see it many times in your project, so you will know what the answer is by the end of the semester. But can you get the key idea? Okay, so we'll introduce a new non-terminal, exactly. We'll introduce a new non-terminal. So we have one non-terminal now, E. We'll have more. That's good. Okay, so what will we do with these more non-terminals? Okay, that would be one solution that somehow the non-terminal will describe a part of the string. It will end with the operator. That's one solution. But we'll do something else, something more systematic than that. But that's a good solution that actually works. And try to see what we do with this grammar here. Imagine that our string is just A. How do we obtain such a string, A? Well, we'll start from E, derive E to T. You'll write it like this. Then T derives into F. Then F derives into A. So this is how we get A from the start non-terminal, E. Now what if we have a plus? What if we want to derive A plus A? Well, we do what? We do E goes to T plus E. What would be our next step in the derivation? We replace what with what? We would replace E with T. So we'll obtain T plus T. We have rewritten this into that. What would be the next step? Both of the T's will go to Fs. And now both of these can go to As. Now we'll talk more about this on Tuesday. But if you see we have introduced non-terminals T and F, we'd actually stand for something. T is a term and F is a factor. And they correspond to different levels of precedence. You know that multiplication needs to be lower in the tree so that it is performed before plus. And this grammar actually enforces this precedence. So that's all you need to understand now. But the key point is that it is not left recursive. At whatever symbol you look at here, say E, it is right recursion similarly for T and similarly for this E here. So this is now not the left recursive grammar. It is friendly to parsers. Okay, so now let's start designing some algorithm. So we do have a grammar and we want to generate L of G, which is the language of the grammar, the set of strings that the grammar can generate. So could we come up with an algorithm that generates all those strings? If the grammar is infinite, then, well, you may not be willing to wait so long to generate all the strings. But imagine that if it is finite, we should eventually be able to generate them. So how would you write a generator of strings from the grammar? You may wonder why I'm asking. Because if you could write a generator, you'll just flip it around just so and obtain a parser. So let's start with the generator because hopefully you will find a way how to, in a really dumb way, generate all strings from the grammar. So what would be a brute force way? It's, by the way, a common problem. Imagine you need to do testing of a sophisticated piece of software. Where do you get the tests? It would be nice to generate the tests automatically. If you could describe with the grammar the set of inputs that the program accepts and should handle correctly, you can then generate those inputs automatically, randomly. So that's a pretty common technique in testing, extremely useful. Well, so how do we generate them? Exactly. So the derivation that we have done here on the right by hand, here we had in mind what we wanted to derive, right? But if we could just go left to right and take the start-down terminal and replace it with one of its productions randomly and then look at what we have here. Oh, here we have T plus E. We could pick T or E and replace it randomly with something, right? That's what we have in mind. And that will work. So here is the algorithm. I'm using this grammar for compactness. So it is left recursive, but it doesn't really matter here. So here is our generator, okay? We call this function, it calls this E, and at the end it prints end-of-file. We'll see soon why we need that. And here it makes a choice and it chooses one, two or three and depending on that it prints A or recursively prints E at plus and another E or three does E times E. You see the correspondence between the generator and the grammar? There is one function here for each non-terminal. This one for E. The number of cases is the number of productions you have. One, two and three. And the non-terminal calls itself because, well, it is recursive. So that's it. So do we believe that it will generate eventually all strings or that for any strings from the language of the grammar this program has a non-zero probability of printing it. It doesn't iterate through all of them but every string has a likelihood of being printed a non-zero likelihood of being printed. Okay, so that's fine. So we have a generator. How can we now... Well, before we go to turning it into a parser what is really important? Often in the parser we don't just want to determine whether that string is a legal string or not, whether it has a syntactic error or not. What is actually more important is to obtain a tree that describes the input. And that thing is called a parse tree. That parse tree is nothing else just a tree representation of the derivations that we have done. So look here. If you look at one possible execution of the random generator which we have just written we start here and now the coin or the random generator have flipped the coin and produced two. So we generate now the second production this one here because we got the random number two and we visualize it as so. Right now it has three children. You generate E recursively and another E and a plus. Then in here we generated the random number one. Here we generated three so there is another E times E and here we generated one and one. You see it here. So you see that the parse tree describes how that string was derived. In our case it describes the random choices of productions. Here we have a production here we have non-terminal E and we chose this production which has number two. We usually don't care about the tree is when we just want to randomly generate those strings but when you are parsing you are learning the string which is the sequence of leaves when you are parsing that thing you really need that parse tree because that parse tree discovers the structure of that input. In the case of the if that else when we wanted to associate the else to the appropriate if it would be the parse tree which would reveal to you how that else is associated. One if or the other. So how does the parse tree differ from the abstract syntax tree which you have worked already quite a bit? So in the parse tree you really everything you see on the input is in the leaves and the internal nodes are the non-terminals. You can read out the grammar in fact from the parse tree because every child, every node relationship with the children corresponds with the production. So when you look at the parse tree and you look at this parent here and these children here what do they do? What production? E goes to E plus E. You can read out the grammar from the parse tree that's too much detail, more detail than you want so we abstract away the grammar detail and build ASTs on which the compilation interpretation program analysis can be done much better. So now we want to turn our generator into a parser and we want to do it mechanically meaning without thinking too much ideally automatically but without thinking too much we'll do. So how would we do it? Here is again the generator. Can someone suggest how we would turn it into a parser? Instead of print we'll do a read we could call it a scan because scan usually means that you look at the next position of the input if it matches what you expect you consume it if not then you say I don't have on the input what I expect let me try another branch. So we'll replace print with a scan so scan will look at the input it will expect that character if it's there it will consume it. So what about the choice? What will that choice do? Are we still going to rely on a random number generator? I guess in principle we could right we could just run it and instead of printing we could read it and after some time the random number will give us a sequence of productions that parse the input and that would sort of work except it would take forever and if there is a syntactic error in the program it would just never terminate right because you wouldn't know whether you're just making a bad random choices or whether there is a syntactic error in the input. So what do we do with that choice for the purpose of reading in the input and parsing? We could iterate through every choice and eventually we sort of have to do it but is there a mechanism that we are relying on in some of our projects that we could just deploy here. So think about the computer science major right you are not just going to iterate you just sort of want to understand what's going on and say I don't even need to implement it because I know there exists a way how to deal with it you use some abstract magical construct so what do we do for choice? You could use a dictionary right? You could but you are thinking sort of to implementation level if you remember the sort of implementation steps we went to describe the prologue interpreter there was this easy step in the middle where somebody did a lot of work for us so we could use coroutine to do that but turns out parsing you can do with coroutines but if I just want to explain the parsing to the algorithm imagine you want to be on who wants to be a millionaire or what is the show at some point you can pick up the phone and call somebody who will give you the answer right so could we do a similar trick here we will just call the oracle and yes the oracle can be implemented by iterating over all choices with coroutine but I want to teach you how to think at a higher level and sort of decouple your thinking from oh I can iterate or there is a coroutine once you start doing that the thinking is too messy and instead you can say I will use this non-deterministic parser sort of relying on this entity which at each step will make just the right choice the choice is clairvoyant it sort of predicts what other choices you have to make in the future when you ask for next choice so exactly we are just going to consult an oracle here so here is the printer the generator and here is the parser and here is also the scan so if the input the current input starts with s you can consume s else abort if the input is syntactically correct then there is no way how we will abort because the oracle is going to give us a sequence of appropriate choices those that guarantee to make the right production so that you consume every character and at the end you can consume the end of file it is extremely a valuable concept to think of this non-determinism because you now understand parsing without having to think about iteration coroutines and in fact how we make that choice now can be done in many different ways but we can talk about it in the second half of the lecture rather than right away so the concept of non-determinism just asking the oracle for a choice is one of the best inventions of computer science we talked about the parse tree but let's talk about it some more so that you understand why we actually parse better we can walk through the parse tree and obtain the AST essentially construct the tree or you can directly evaluate the program if the program does not contain loops and it's enough therefore to walk over each node once so here is a slide that shows how you could given a parse tree shown on the right you could actually evaluate the program so again the inputs are in the leaves here you see every production every edge is one production so the three edges here are also one production which one E goes to E plus T and you just propagate these values up and here when 4 meets with 5 you perform an addition you obtain 9 and on the root you get the value of the expression it's pretty much like on the AST except the tree has many more nodes some superfluous when it comes to the AST alright and you can go directly to the parse tree as described and make it generate the parse tree so look what we've done we still have sort of the parse routine which calls the start non-terminal and then at the end it expects to have read the entire input here is the procedure for E we ask an oracle which choice to make it reads A, it returns A and then it sort of parses the left hand side the right hand side it expects plus and it returns this node of the AST so what we are doing when you run into A you return an A leaf when you run into plus you return a plus and then another A here you construct this node and so on so the structure is before except we are now returning the parse tree to the result so how could we now now we need to get to the business of implementing the oracle of course we don't quite have the oracle you could perhaps crowdsource the business of oracle to the net and just sort of pose the input string crowdsource it on mechanical torque and say well tell me guys what decision I need to make here and here it would not be a very fast parser I suppose also quite expensive by the way the parsers that we are running for you on the on the Google cloud cost about 2-3 bucks a day I think that the crowdsourcing parser would cost more so we really want to find a way how to run it efficiently without having to really ask people to act as oracles so we could implement the oracle with coroutines through iteration but we have already built the prologue interpreter which does essentially act as an oracle right it does the search it finds the right match so that the goal can be proven to be true so essentially we already have built this power of the oracle into the prologue interpreter so we are not going to use for the lecture coroutines or anything we are just going to write a parser in prologue and the prologue interpreter will serve as our oracle so let's go with this grammar it is not left recursive because prologue even though it has an oracle itself is not all powerful and it would actually loop forever on a left recursive grammar we'll see soon why and we'll write it like that so we'll have one predicate E and the way it will work that we'll pass the string here and at the end of the pars we expect an empty input so you could think of it that this is the input string and this is I should say E and this is the output so maybe I should call it the rest of the input so here this corresponds to the production E goes to A so if I pass here a string maybe A plus A what do we expect on the output from here so if you remember this is the prologue notation for lists this will separate the head from the rest of the list and this out and that out are the same variable so if I make a query E A plus A out what will I get here assuming we only have this one single rule we'll get plus A exactly so the output will be plus A so if you see what's going on in this rule this is one to one correspondence to what we had before in this non-deterministic parser where we had a scan the scan checked for A if it was A it consumed it from the input and left the rest of the string untouched the input string so this rule here does nothing more than scan A if there is A on the input it will strip it off and return the rest of the input this way so let me write here this is a scan A now how about this production how would you do that now we do need some recursion to implement that production so again keep in mind that we are doing nothing magical we are taking our functional non-deterministic parser which we understood a second ago the one with the switch statement and then we are just turning it into prologue so that prologue does the oracular decisions for us so anybody who wants to write down what this rule would be you could match multiple inputs you could match say A and a plus absolutely you could do that so you could match these two at once and now you need something right so we actually need that exactly oops well I think I let's use what's here well what are we doing we are simply checking whether on the input we have A and plus this is the rest of the input after A and plus have been stripped this goes here and now we are calling the parser recursively it will parse whatever starts at R and return the rest in out and this goes back here now how do we call the parser we just want to give it the input string and we expect the whole thing to be consumed so this here corresponds to scan end of file right the whole thing is consumed we don't have a special marker for end of file but we expect that the whole thing is consumed so now in three lines we had what we had before in more and it does the entire parsing of this grammar and I could run it for you but if I just run it like this it would tell me through okay this is kind of beautiful that the power to make non-deterministic choices is to a large extent hidden in prologa and you can nicely exploit it so let's look at we just talked about how this works so this is written here for you at home to read now here is the parser for the entire grammar remember how we took the grammar of E plus E times E and rewritten it to refer to terms and factors okay so here is the entire prologe parser for that again here is the f part here and here is the this production and here is that production but again the entire parser is right there now could we extend this parser to also build the parse tree remember we did that procedure we do it now in prolog so that the result is a parse tree so currently the rules get the input they will consume something from the input and return the rest now we want them to return exactly the subtree just like before and the way we do it in prolog is let's look at the base case right when you find an A on the input you just return the A this will become the leaf of the parse tree now if you do the production say the star production here you get the subtree to the left of the star here the subtree for the right of the star is here what are you returning you are returning a new node of the tree with this tree hanging here this tree hanging here and this is a new leaf and indeed if you run this parse you will get that this is the flat representation of the tree now one more quick look for you have a look at the program and tell me what do we need to change to produce an AST rather than a parse tree is this a simple change that we could do remember the AST somehow omits some of the nodes that are in the parse tree is this a small change now if you look at this here if you look at this production here we do not want to have any node for that and currently we actually do when we take this production it happens in this line we do create a new node and attach the subtree to it so how do we turn it into AST essentially see here whatever tree I get here I just propagate and again tree that I get here I just propagate for the star rule I create a times node there should be a comma and this should be something like a plus I'll need to fix these mistakes so this should be a plus and this is okay but this needs a comma and here indeed if you run it you get an AST for that input string well what do you think is the time complexity of such a parser given a string of length n how fast does it run so I have written it using the m-language from 61a or alternatively in prologue here we are asking an explicit oracle here prologue is going to make these choices for us 1, 2, 3 and for in the worst case each part of the input it needs to make a decision am I going to go here or here or here of course it cannot really make oracular decisions it needs to try them and then when it gets stuck it will backtrack and try the next choice so turns out that the complexity is exponential because you need to make one of three decisions and then another one of three decisions and another three of decisions so the parsers could run in exponential time you could fiddle with the grammar to make it fast but it could be in the worst case exponential which is not that great so I'll show you how to do it in a polynomial but let me say that this parser here that we saw is called recursive descent parser if you remember the calculator that we looked at in the second project that is written as a recursive descent parser not in prologue by hand in python and you should look at the code because it works exactly as the prologue parser we have here but written in python why do you want to know that because each time you have a small parsing tasks a small configuration file that needs to be parsed this recursive descent parser is the way to go it's an easier way to implement small compact parser in fact languages like javascript in the firefox browser have lexers parsers indeed which use exactly the same recursive descent algorithm and they are fast enough and they achieve speed by making it to decide between the alternatives they rewrite the grammar such that each time you need to make a choice you can just look at the next symbol and say oh I know which way to go so you don't need to try something and then backtrack and try another one but let's see whether we can do this parser more efficiently than exponential time we'll do it in polynomial time and we'll develop what is called cyk parser does anybody know why the parser is called cyk it's sort of named by three different people who invented the parser independently so it's sort of the easiest systematic parser that you can build it's so simple that you could invent it yourself and in fact a little bit of guidance in previous years we could just invent it during the lecture we are going to do something cooler today rather than just inventing it we are going to show you how if you write it in prologue just the right way we can get this polynomial time so what we'll do there is a thing called datalog maybe you'll learn about it in the database class it's a well behaved subset of prologue so certain things you rule out and then the prologue language becomes smaller, not everything is allowed it behaves better for one thing it behaves now all of a sudden like real logic I don't know if you looked in the art of prologue why prologue is a mess rather than real logic it's an interesting discussion why the meaning of logical not in prologue does not really mean not not always means no apparently but in datalog it always does and so what is datalogue it's a prologue where you disallow certain things one of them is you cannot have these nested compound queries or rules that's a little bit of a problem for us because we implement the lists essentially with these rules our cons is like this F1 so the cons is not allowed in datalog so lists are out and then you only allow so called range restricted variables so when you have a rule like AX then this X must appear here so that you could compute the value of X on the left hand side from the value on the right hand side and then there are certain restrictions but for us it's enough to say that let's just don't use negation if you don't use negation then we are definitely within the datalog subset and you can read more on this really well written Wikipedia page but why is it interesting for us first of all it has predictable semantics not all prologue programs terminate if you gave them a left recursive grammar they would just keep expanding the goal keep expanding the goal and they would recurse forever if you give prologue a rule like E N and now you give it the same input whatever goes here so this is E what would prologue do you call it on this goal you match this goal and now you recursively expand this goal what would you do it would just do it again and again and again until it runs out of stack so that's the reason why the left recursive grammar does not work in prologue because the same value here the same list the same input is passed here without consuming a character on a right recursive grammar remember there is always some part of the input that you can scan you can consume before you call yourself recursively and so the input is shrinking but not on the left recursive grammar so with datalog all evaluations terminate so there is no question whether the parser will get stuck on every grammar it will actually evaluate it and then we don't need to do this backtracking business we can nicely evaluate it bottom up by essentially filling in a table like in dynamic programming and it will finish in polynomial time in n-cube so in fact datalog is so nice that we can use it to explain the parsing algorithms we will use in the rest of the semester and we'll say well the CYK parser is really just the datalog version of the prologue recursive descent parser so what we'll do we'll take our parser here massage it a little bit to be in the datalog subset and it will all of a sudden run not only in polynomial time it will run for any grammars left recursive too and then the early parser is just some optimization on the CYK I won't quite explain what the magic transformation is in general but specifically you will see how you can make it much faster but the bigger lesson for CS164 is that if you take a language and you throw out features from it and you say you can only do this but not that all of a sudden the language is easier to understand implement imagine scheme from which I remove recursion it would be much easier to write an interpreter for it if there was no recursion it's a different story that you couldn't write any programs but you would not need to have a stack for one thing so the ability to subset the language to a smaller set of features is extremely powerful at any time in doubt throw it out and the datalog is a nice case because we are restricting it quite a bit but it turns out to be enough for our parser so let's see so is our parser actually it is in prologue clearly I ran it this morning but is it in datalog so does it have no negation there is no negation in those rules right is by the way the entire parser for this grammar is it range restricted it is because this out value is computed from the right hand side r is computed from here so yes we can compute the left hand side values from the right hand side which is why you can compute it bottom up from ground facts rather than backtracking top down so that's good does it have compound predicates yes the lists here this is essentially a con so we need to do something with those lists so we cannot do it completely automatically but if we have an idea how to rewrite this parser so that it doesn't use lists we create ourselves a CYK parser we can add your initials to the neg so how do we get rid of those lists any ideas so you essentially want to break the list such that every element of the list is is a separate parameter but the list could be of arbitrary size so I'm not sure we can do that transformation so what if I give you add the ability to sort of index into an input array imagine that I do give you that in datalog that's actually easy to do that you read in an input we put this input into a big predicate something if we have an input A plus A plus A then maybe we could have a predicate called input 0 is A input 1 is a plus 2 is an A plus that allows us to access all characters of the input without having to use lists so now we do have access to the input can we now with this representation of the input so we do have one representation of the input can we use that to rewrite our parser so that it is in the datalog subset and remember the only thing we need to do is get rid of those lists so I think we are on the right path we are storing the number that is the rest of the input that turned out is not quite yet it could be maybe enough for that maybe yes let me show you what the solution is it is pretty much like we are describing so we are now going to make the E predicate not to use string coming in and the rest of the string coming out predicate E will compute 2 indices and they will be such that this E, I, J is true if this segment of the input from the I symbol to the J symbol can be derived from E so if I have here an input and here is I and here is J if this here is A plus A then E, I and J is true why because the segment of the input between I and J is derivable from E so now we are representing the input by saying if this is I, this is J anywhere in the input if I can derive this segment here from E then E, I, J is true clearly if this here is a plus then is it the case that E, I J plus 1 is E, I, J plus 1 true then what follows it is a plus can I derive A plus A plus from E it's not a string derivable from E it's not an expression there is this hanging plus so this would be a false okay now let's look at the algorithm the first rule again how many rules are we gonna have again like two again this is the base case what does it say that this is true when what when on the I's position of the input we have an A this corresponds to this production right so does that correctly implement the production if you find a chunk of the input just one character that is an A is that A also an E it is derivable from E I have a few alternative ways to explain it now let's look at this production so now I call it with I and J that could be arbitrarily far from each other not just next to each other so I and J and let's see we are asking now does there exist some K such that this part of the string is derivable from E I to K minus 1 the Kth one is a plus and then the rest from K plus 1 to J is derivable from E essentially what we are asking here is that if this is I and this is J we are looking for something derivable from E here then we are looking for a plus here and another thing derivable from E and this here is the Kth index so this part here is E and this is E okay and similarly for the star now I have two ways how to explain what's going on here the first one is how would we evaluate the program can we go bottom up and start from the simplest facts and then build more so if I give you an input let's now I'll write this down by hand and if I do this this is 0, 1, 2, 3 and 4 and let's make it a plus a plus a okay if you look at just this rule what facts hold true right away given this input we want to do an iterative algorithm we see which facts hold right away and then given those we deduce more facts and more facts and more facts I'm gonna represent them just a second are you asking about the first rule here yeah let's assume that the input is broken into a single character so every element of the input contains just one character was that the question no as a thing of this one okay good question so imagine this is I here and this is I plus 1 this rule here is really asking about this element here so I plus 1 is indexing to the next one yes it should be and indeed I had it correct this morning and I screwed it up thinking that I did the right thing so this is the one algorithm I did not run alright good excellent so which facts hold right away can we put into this table information about which facts hold given this first rule so let's say this is I and here is J so 0 1 holds okay 2 3 so these indeed hold right that's what we know given the first rule how about now given this knowledge can we use this rule to learn some more so essentially what do we know we know that this is an E and this is an E this is what we have learned in the first round right we know that this can be derived from E and this and that can we now learn that this whole thing here is an E we should be able to see that this is an E then there is a K which is a plus and then that one so we do have 0 1 is here now do we have a plus number one position yes it is here right there is that plus and then we have 2 3 which is here and therefore this here is an E as well by similar reasoning we can show this one right or did I miss it by a point 0 3 it should be 0 yes it should be 0 3 absolutely and it should and then the E that corresponds to this one here would be 2 5 right and then the next round we take the information about these two and we put them together and this is the knowledge that the whole thing here can be derived from now this is all written here so you can step it up by hand again but this is a more interesting way to think about it and indeed this is why it is on the T-shirt because you will see this algorithm again and again when you work on project 4 and this will make it really clear this graph representation so when I take an input I can represent it as a graph what kind of graph a chain graph one edge for each element of the input right and now I am going to go bottom up and perform what is called the reduction so when I see an A I see oh I see there is a production that goes from E to A so I can take this A which is the right hand side of this and reduce it to its left hand side so I can now place an E here and another E and another E and I place now we see what we see E plus E is that also an E? yes because we do have another production which says E is E plus E and we see the right hand side right here we see this E here and a plus an E next to each other which means that the whole thing from here to here is derivable from E we are reducing this right hand side and we do the same here and now we see what? you see one E here and at times an E that means that the whole thing is derivable from E or with animation you do this in the first step then once you have these two E's you can do that then you can do this and the fact that we have placed an edge that we managed to place an edge from the beginning to the end is labeled with the start non-terminal signifies what? signifies that we can derive the entire input from the start non-terminal of the grammar but we built it how? we built it bottom up by looking at the individual characters and reducing them to the left hand side of the non-terminal until we were done so I'll leave you with that you can look at the slides and see how what we could call CYK graph you could reconstruct the parse tree can you look at the graph and see the parse tree in it? there is a mix-up between nodes and edges so it's interesting but yes the parse tree could be read out from here and we'll continue on Tuesday I'll tell you about the early algorithm and then on Tuesday you'll get a homework in which we'll give you an early algorithm running but extremely inefficient it will run like N to the fourth it will probably parse 20 characters and you'll have to optimize it to get it down into linear time so that you can actually use it as a parser in your project essentially it will be an exercise in designing good data structures ok thank you