 So, we will start with the idea of continuation and then talk about the thing called continuation passing style and with time permits we will also look at implementing interpreters using continuation passing style and also try how we can implement called cc in your own like continuation in a language. So, yeah, so this is the idea the brief outline of the talk idea of continuations and we will spend a lot of time there and so we will use scheme. So, the scheme has this thing called called with current continuations. So, we will spend some time with that and we will also talk about tail call optimization already we there was another talk about tail calls and how to do tail calls we will see yet another way of achieving tail call optimization then if time permits we will write a toy scheme interpreter and implement the same using CPS and also implement called cc. Anyway, I will put the code out there if time is in sufficient little bit about me I am basically an electrical engineer mostly self taught programmer work for this company called conquer technologies where we are trying to scale the back end using some functional programming techniques and have contributed to a few free software projects. So, very brief introduction about scheme we will be using scheme here I think we have already looked at closure code and also other lists code in the other talks, brew states talk also had I think the more tense talk had some scheme code there. So, scheme is called by value it is eagerly evaluated and it has expressions. So, first the first thing in a expression is assumed to be a function and rest of them are operators and you can override that with the code and change code to data and back to code and there are very few built in constructs and other thing that we take for granted which is lexical scoping first appeared in scheme I think it first appeared in Algold, but yeah it is a one of the first list that got lexical scoping right and then functions as first class and usually these things are covered in most of the introductory programming courses that use list, but macros and call cc those are considered to be some of the difficult parts to understand in scheme. So, yeah one of the thing is what I am trying to cover in this lecture the other thing is that my goal for this talk is that you go back with some understanding of continuation I had a lot of difficulty understanding continuation. So, my goal is that you go back with some intuition about continuation so that you can look up you can look up by yourself and understand more and at any point of time place interrupt and ask questions during this talk or later or any point of time yeah. So, yeah this is the famous quote that is used at the starting of the r5 rs standard the scheme revised report on scheme programming languages should be designed not by piling features on top of features, but by removing weaknesses and restrictions that make additional features appear necessary. So, continuation is one such feature it is like one of the primitive things on top of which you can implement a lot of other control concepts we will see at least one or two of them ok. So, let us say we want to add two numbers very simple we just want to add two numbers. Let us say that you are reading the first number from somewhere and then you are reading the next number you are adding it and displaying it. So, read number could be so this could be say in a web scenario where you are asking input say toy example say you are asked to input first number on a web page you have to press a submit then you are asked to enter the second number again press submit and gives you a number two or whatever number 42 or whatever. So, when you press when you enter the first number and when you are asked the second number the web server has to remember that you have already entered the first number. So, there is a computation suspended out there which already has which knows what it needs to do once you have entered the first number right. So, this you can encapsulate it in a function that what it needs to do once I have read the first number it is nothing but a function takes the first number as a parameter then it reads the second number and thus the addition and gives you a result. So, continuations are nothing but rest the rest of the computation that you need to do but rest after what so after some context that is you have done some work now. So, once you have that value what you need to do to complete the rest of the program that is what is continuation. So, let us take the first expression there that is you are multiplying 3 and 4 adding it with 5 and giving back the result. Now, let us look at 3 when you are looking at 3 what is it what is it that you need to do to complete this computation which is nothing but at the point of 3 we have put a question mark there. So, given 3 I need to multiply it with 4 add it with 5 give me back the result. So, you can that is what is shown in the next 2 expressions. So, you if we is 3 then we compute it using a let expression there. So, let is nothing but syntactic sugar over lambda right that is you have a let you can transform it into a lambda expression it is a function application. So, once you are given 3 you just apply it and get the result. Now, if you really see that the function the first function the function the lambda expression. So, that is that is really the continuation that is what you need to do once you get the value 3. Is this clear this much of it is clear anything unclear about it. Yeah, so, forget about compiler right now think about a program execution in an abstract way. That is let you are sequentially executing a some program at some point what is the rest of the program can you encapsulate it in a in a function that is what continuation is continuation is nothing but a closure that takes one value as a parameter that value is nothing but the computation we have done as of now once I am given that value what do I need to do rest that is encapsulated by this function and that is what continuation is above any other questions is it clear whatever is whatever is explained as of now. So, now, if you see any program implicitly has a continuation only thing is that most of the languages are not explicit about it does not languages does not allow you to capture continuation into a variable. Continuations are not first class in most of the languages any C program or whatever program an assembly program you have continuation that is you always have a rest of rest of the computation right. So, let us take another example same the same expression. Now, let us say we are evaluating that expression before evaluating what what is the continuation of that let us say you are inputting that expression into a ripple what is the continuation of the program you have to just return the results right that is the continuation you just have to return the value. So, the continuation of that is an identity function. Now, let us say you are looking at the inside expression star 3 and 4 multiplied 3 and 4 what is the continuation of at that point it is once I get 3 and 4 multiplied I need to add that value with 5 and return the result that is a continuation of that program at that point. So, you can also say that lambda v plus v and 5 is the context of star 3 and 4 that is what do I need to do to get the entire result given that I am looking at 3 multiplied with 4. Now, let us go still further recursively now you are looking at 3 looking at the star 3 and 4 that is you are looking at it at the evaluation order from left to right let us say. So, when you are looking at 3 same thing repeats. So, I hope you get the idea. So, we are these continuations are only up to the printing of the result at the ripple if you really see the continuations go all the way up to the scheduler right. The scheduler is what is actually calling this program and actually running it. So, rest of computation is going all the way up to a process schedule yeah that is only if the order of evaluation yeah yes. So, if you are converting a program into a continuation passing style or into like explicit continuation then your order of evaluation is explicit any questions this point yeah yeah. So, it never returns if you really see you are actually calling the continuation of the program. So, here if you see the big expression at the first right continuation of that is nothing but a identity function right. So, given the value of that you. So, we are going only up to the ripple here actually there are lot of other things happening after that, but we are not capturing it here. So, there are other things like delimited continuations where you can actually put like you can focus on one portion of the program and say that this is what I really care about the continuation up to this point you could do that, but this is not delimited I am not covering delimited continuation here I am assuming left to right. So, if you actually convert a program into a continuation passing style you have to assume some or you have to make it explicit without that you cannot yeah. So, that is one thing I am assuming here now if you really see this is an alternate model of evaluation or it is an alternate model of thinking about how to do computing right. In a normal Born-Human architecture in a normal C or any Born-Human program execution model you have a stack and let us say function A called function B function A will have its return address pushed into a stack then it calls B then it returns back to the stack and pops out the return address and jumps into that value right. So, we are essentially modeling the same thing with continuation the stack and continuations can really be thought of alternate ways to think about the same thing yeah. So, some more examples to just drive the point. So, what is the context of chart 3 and 4 in this expression. So, as a first step what we do is we replace the expression of interest with a question mark our expression of interest is chart 3 and 4. So, we get that second expression now try to simplify that expression do exactly what we discussed last time that is you wrap the whole thing in a lambda that is your continuation. Now, 0 5 is not 0 right you simplify that. So, the if condition will fail. So, it will take the second branch and wrap it in a lambda. So, that is a continuation of this expression clear yeah one more example. So, again what is the context of chart 3 and 4 first step again yeah I simplified the first step the value as well here in the first step. So, here multiple things are happening. So, we need to wrap it in a begin. So, n is 1 we know that n is 1. So, it goes to the second branch and it has to return the value of n. So, actually I have to return the value 1 there it is a typo there. So, begin so that is what happens there now you wrap the whole thing in a lambda. So, that is a continuation as far as chart 3 and 4 yes the continuation will be different in that case. So, if n is 0 what is what is the rest of the program it will take the first branch right yeah first first display it will take right. So, the continuation is different in that case continuation means rest of the program. So, rest of the program is going to be different in that case yeah yeah agree yeah yeah context of 3 and 4 is irrelevant. You can you do not have to evaluate you can actually find continuation for all possible branches. So, this actually are just for thinking about continuation what is. So, this is not how a compiler will do what a compiler will do is it will do source to source transformation before it actually. So, during compilation time CPS is one path that is it will pass the continuation at every point we will see some examples I am coming to some examples there what is the point here. So, point is that we make control flow explicit right and other control operations like loops non-local exits coroutines or generators you have the yield in python and things like that right. See you can represent all of that in using this primitive you can think of continuation as a primitive to implement rest of the control flow operation. So, that is again if you see scheme that is a that is a theme in the entire language where you have things that can be built from bottom up. It is you have small or very orthogonal primitives at the bottom and you build the rest of the things using these. So, we will see an example of a non-local exit we will not have time to go through generators and coroutines but we will see non-local exit. So, third point there is that continuation is present in every program if you see nothing but rest of the computation right but not all programming languages allow you to capture it in a variable go back to that continuation and things like that. Very few languages support it ML supports it standard ML that we saw Basker talk scheme supports it yeah. So, there are a few such languages. So, that brings us to the construct used in scheme called call CC or call with current continuation. So, call CC has a very weird API but essentially let us look at this example again because we had been thinking about this for a while. So, again come back to the same expression char 3 and 4 that is the context we are looking at. Now, let us say you want to capture the context of char 3 and 4 in a variable this is what you need to do. So, you wrap char 3 and 4 in this expression called call CC. So, call CC call with current continuation that the current continuation at that point of time is passed in a variable called K. Now, that K itself is wrapped in a lambda. So, call CC's argument is a function which takes another function K. So, it is very confusing you can there is another concept called let CC that takes just K. But essentially K is nothing but the current continuation that we had been talking about that K is nothing but the lambda expression that we have written there lambda of V star of V plus that is given the value of 3 and 4 multiplied what do we need to do next that is bound implicitly to the value the binding K. So, now we just return it we are actually not using K. So, K has a scope inside the lambda it has got the scope only within the lambda expression we are not using K there. So, you can actually use K K is nothing but a function K is nothing but a closure which takes one value as a parameter. So, you can actually apply K to char 3 and 4. So, what do we get it will just behave like the original expression plus of char 3 and 4 add with 5 it will just behave like that. So, what does it do it will just return 3 and 4 multiplied. Now, let us do something interesting define a global variable char K star that is a convention in the list world where globals have this earmuffs you put char on both the sides. So, now what can you do within the lambda you can capture K which is current continuation and set the star K star with that value and it returns back. Now, you can actually call let me try to why does it not show here it shows on the screen oh you have to mirror it does not show anything here yeah it is there, but it is it is not showing the screen it is not it is actually not detecting that device otherwise it is okay. Oh yeah that is a possibility and try to move it into this okay yeah I think it is okay I think I will just talk it is yeah I get this, but I do not get any other application let us say I want to show the racket I just on the star K star we have actually captured the continuation and the continuation is nothing but this function there lambda function lambda of v plus v and 5 that is the continuation. Now, you can actually apply star K to any other value you can pass say 42 into that if you pass 42 you will get 42 plus 5 instead of star 3 and 4 so it is like you have captured the rest of the computation at that point of time and you can go back to that point of time that will also be 17 now K we are not using K is not identity K is the continuation at that point of time current continuation we are not using K there it is just returning that value this expression when you are saying it will compute it it will complete it computer I am not able to show the racket ruffled control goes to the lambda control goes to the expression inside the lambda yes it just returned the last value of it just returned you have a choice to use K and do something interesting with it yeah you are exactly you are placing a marker there you actually captured the marker in a variable K that is all it is using that exactly because we are showing it we are able to come back there just a marker exactly so here there are there is no difference between this the value returned here and the value returned here why because K is the same if I had used a different so K is what K is current continuation current continuation for this expression is what is a lambda function right it has to take the value and add 5 returned result here also it is the same thing so what are we doing here we are essentially calling this lambda function lambda of v plus v and 5 with the value star of 3 and 4 yeah yeah interesting things will happen the thing is here that the continuation K is nothing but this function at the bottom we are now calling that function with star of 3 and 4 which will return the same value 17 understood now if you use another value there let us say you use another value we are actually jumping to the we are actually calling the continuation with the value star 3 and 5 yeah it will not do it will not do it is actually jumping into the continuation that we have captured in K it will not come here that is an important point so you actually you never returned from this value you have actually you have passed the control to another point of time I will show another very interesting example I expecting to spend at least 5 minutes on that we will come to that so yeah so this is actually same thing we captured the value K into star K star we apply to some other value like 20 you apply you get a different value because rest of the computation you are applying to a different value yeah so one one simple application of this let us say you want to I should have called the break as returned here you want to scheme does not have a return expression like in C right you have returned some value now let us say you want to implement return in scheme I think instead of break let us assume return but let us let us say that you are you are multiplying a bunch of numbers recursively and let us say you have if you have if you encounter a zero at some point of time you do not want to just multiply blindly you want to return immediately without doing any further computation that is what you want to do so if you just do recursively you still get the same result but you will end up even if say first value is 0 you have say 1 million numbers you will still end up multiplying the whole thing right now let us say you want to return the moment you enter you encounter a value 0 so what do you do you capture the continuation of outermost continuation you are you are using call cc right at the outside right the first thing you do inside the function is capture the current continuation in a variable break now you look at care of LS the list if it is 0 you call break so it just returns the value 0 because the current continuation that you have captured is nothing but it is just exiting out of the program else you do the usual stuff is this clear this example is a non-local exit yeah yeah it is nothing but a closure it takes one value as input it is nothing but this yeah because break what is break break has break is nothing but a function that represents the continuation at that point of time so what does it do yeah it just did a go to there yeah it returned actually so what does it return it returned yeah come to that what is break what is a break value what will it be it will be an identity function right it is an identity function it is not some lambda it is identity function because we are capturing the break right at the outermost part correct that is what we are returning break 0 returns 0 we do not want to do that that is why we did that right the moment we encounter a 0 we want the control to jump to the place where we are exiting the program we are actually calling the break we are calling the continuation with the value 0 the continuation is nothing but a closure it is a lambda yeah takes one parameter now what is that closure in this case it is an identity function right it is true yeah it is a closure no if you yeah right yes yeah break 0 returned 0 what is it used for that is nothing but the continuation at that point of time right yeah yeah it will not use the else part moment it encounters a 0 it will just return it will jump out to the value it will jump out to the end of the function because break is what break is the current continuation when we started it is the outermost part right if you had put break somewhere else if you had used called no no let us say let us say we are the first example here first example that is it does not have a 0 okay yeah so if you pass the continuation along so you are calling the product with a plus 1 of product something right it will return 121 because the continuation at the break break captures the continuation right what is the continuation given the product I need to add 1 you are not calling but it actually returns that value right so it it it is the same example as this that you are talking about right like you have some value instead of the star of 3 and 4 we are doing that product there but yeah let us say you wrap that call you see inside the product and you are adding some value 1 right so if you just return it will just return that value if you do not use break there it will that is the continuation at that point right so break has actually or k k has k has captured this continuation lambda of v what do I need to do to complete this expression that is what is the context around this expression context is nothing but you need to add some value to the result and return that value okay so nonlocal exit now this is a bit complicated example if you let us spend some time on it if we really understand this I think you can really claim that you really understand continuation now so you have a you have a let expression there is a binding x all it does is called cc lambda of k which is a current continuation k it returns that value k that is what the the x x is getting now in the body of let all you are doing is you are calling x you are invoking a function x with another function lambda ignore high that is a function now this returns high think about it for a minute do not think do not see the rest of the part try to think only at the first part of the expression please do not see the rest and try to see if you really see it so we have captured the value k which is nothing but the current continuation and that value goes into x it prints the value high so to break it down again go back to the same thing that we talked about the steps that we took what is continuation k k is the current continuation at that point right wrap it wrap the whole context of that in a lambda right there is a first step you can do so you wrap the whole let expression remove that calls you see because that is what you that is the context right k is returned context is returned to x so continuation is nothing but a lambda of v assign the value v to x invoke x now what do we do now we invoke that continuation this continuation lambda of v the whole thing now we invoke this with another function lambda ignore high at that point what happens x gets bound to v which is this argument lambda ignore high once again right x gets bound to lambda ignore high and then we invoke lambda ignore high lambda ignore high and it returns high understood this what somebody said no what what exactly you did not understand okay I will explain once again what is the continuation k to call cc or call with current continuation means you are passing the current continuation what is the rest of the computation at that point into the variable k or into the identifier k so k is bound to the current continuation now you are actually just returning k there inside the call cc so that value the current continuation of the current program that I am running gets bound to the identifier x now I am invoking x with a function lambda ignore high now at that point what happens this continuation the second expression there gets invoked with the argument lambda ignore high now what do we do there inside we are bounding the argument that we are passing to x so x gets assigned another value again and then we invoke lambda ignore high with another lambda ignore high and it just returns high understood ignore is a is a parameter right this is what x get got bound to lambda ignore high right yeah now you are just invoking that here ignore is going to be ignored yeah yeah exactly it will be lambda ignore high lambda ignore high and we are ignoring that function returning high yeah exactly yeah you do not need any other special constructs yeah exactly it is a nothing but a here so ignore gets assigned to this value it is an anonymous function right lambda ignore high now you understand okay this is the simplest thing you just apply this function to another function now this whole thing gets assigned to it but we are not using it then it would have invoked this function yeah but you can do some variance of this let us say if you put instead of lambda ignore high if you put something yeah but I hope how you reach there to the last statement I think that is the most important thing how you actually invoke last statement probably not the important thing but how you actually reach there what is the continuation what is the value of k so you can easily break it down using this technique that is you replace the area of interest with a with a identifier wrap the whole thing in a lambda that is your continuation yeah so now now let us say the hello world of functional programming factorial example that is how you write a normal factorial example right so we have 15 more minutes that is how you write a normal factorial example that is if n is 0 you return 1 else you do the example that the continue the tail called optimization talk talked about so you invoke factorial of 5 then you have a triangular like execution model that is you keep you have to store the previous value and yeah it is a normal execution now the case of tail called optimization or if all you if all your calls are tail calls then you get a very different execution state that is you can you can call this as accumulator passing style whereas where in you what you do is you you also have some identity value in an accumulator and you keep you keep adding your accumulating the result essentially folding your value in intermediate values and computing the result so this has a very flat stack or if you can the the fact acc is can be it can be just a jump because you really are not you do not have to store anything to be done after we call the fact at acc right so I think you you will start seeing some very painful so what do you do fact slash k has a continuation k as another parameter so one more parameter it takes at k when you invoke the fact is nothing but identity function you just have to return the value now what do you do now k is continuous actually this k and that k you are confusing I think I should not have used k in the invocation instead of you yeah yeah it should be I just an id it is an identity function yeah I should not have yeah yeah so k k at that point is the current continuation you can you know k always takes one parameter as input that you invoke k with one parameter you can return k k is first class right it is a it is a closure you can return pass it around do whatever you want yeah when you invoke you need to invoke with one parameter takes one parameter yeah yeah sure sure yeah I think it should not be k like I think you should call it n or whatever that is the continuation of this program no they are not continuation yeah yeah that is identity exactly exactly yeah yeah yeah that is that has nothing to do with continuous it is a wrong usage so what is happening here so we converted our previous simple factorial function here right into into a continuation passing style so what do we do is first thing as we add a new parameter k with represent continuation now if n is 0 we have to invoke the continuation and return the value 1 now take the second example so the second branch of it if n is not 1 or if n is not 0 what do you need to do given the value of factorial of n-1 we need to multiply that result with n and return the value right that is what we have in this right given factorial of n-1 multiply it with n return the value that is exactly what is done here you create a new continuation inside the second invocation the invocation of fact slash k so you are actually creating a new continuation and that continuation has to what should it do it has to given the value of fact n-1 multiply with n and return the value so that is what what is happening here right if no you should say k you should say k 1 that is where it unfolds exactly actually this program does not return it actually passes the control to k this program does not return all continuation passing cell programs never return this is clear okay some more examples we have 10 minutes I think I cannot show the interpreter program with this laptop I can try with another laptop yeah continuation no we actually use it in an assignment both of us Anand and me together in a while we wrote a python interpreter in scheme yeah in racket we actually used it to implement generators for example yield operator in python so the compiler if you have used any functional language this is a one of the first step that happens in any functional programming language compiler Haskell or scheme or any anything so you can actually implement it in scheme with a macro CPS can be source to source transformation so it is like a perfect fit for scheme macro so let us take a sum of a list another example very simple example so it takes a continuation now count and again same thing so given the so if you say the first one without continuation given the sum of the rest of the list what do I need to do to complete it so you see that the first part of the program is not in continuation passing cell and not into a tail call position whereas if you convert the program to a continuation passing cell you always get a function invocation in the tail call position that is another property it is like a side effect of converting a program into CPS side so CPS was not invented for getting tail call optimization that is like a nice side effect yeah so some more example again in lower level example like Fibonacci program okay so you have a you have a generative recursion here and again so here two invocations of Fib is happening so when you convert that into into continuation passing style the those two invocations are sequentialized or they are called in sequence so the first one the k one is the continuation of the first Fib that is given the value n what all needs to happen to compute the Fib of n given the value of Fib n minus 1 what do I need to do to compute the value Fib n so now inside that you are again we have sequentialized the whole thing now given the value so you still have to figure out Fib of n minus 2 there right so given the value of Fib n minus 2 so given these two values we just have to sum and return the result so the first one first program was not really very sequential program but whereas the second one has made it really sequential some more examples you have say given a value you need to find out whether it is odd so the first one is without continuation second one again so given the value X of odd if odd of n minus 2 if that value is given then you invoke the continuation again it is the same it is the same thing this a bit more bigger example let us say higher order function filter so again first part how you do filter without continuation passing style that is the given list is empty you return empty that is you return an empty list now you look at each value of the list if it passes the predicate test you count that value else you do not count it and keep looking at the rest of the list now how do you convert this into a continuation passing style you have multiple invocations of functions happening right so if you really want to so all the functions in your continuation passing style program should be in the CPS state except the primitive like plus and minus and the lowest level program everything else has to be CPS otherwise there is no point so we need to CPS the pred function as well here so pred also takes an extra parameter k its continuation so again we just return empty with the current continuation k or else we first have to do pred that is you look at the you look at the first value now what is the depending on the pred result you are going to take either of the branches so the continuation at that point is very clear you are going to take either of the branches given the value result of pred and it is going to take one of these branches which is also converted to given given the value pred then what do I need to do to do filter k I think this line should be called the pred slash k everything has to be in the CPS here so as we said can be done using a source to source transformation generally that is what is done in every compiler but we can also do in by hand I have the program I may not have time to show but I will post the code somewhere these are some resources that you can look at SICP 4 chapter has has a continuation passing style program but the first time the name continuation passing style was invented was in the second reference the rabbit the first compiler for scheme by guys seal so that had so he invented the term continuation so and it is another nice book the third third reference which is out of print it is one of the best reference on continuation you can find it in Amazon marketplace or somewhere it is very nice book and then programming languages applications and interpretation one and two both are extremely nice preferences and the lambda ignore high example is from the scheme programming language Ken Dewey's book and there is also a very nice Google hangout recording by Will Bird so he has a extremely nice continuation example yeah with a small parting thought this is the end of the question so yeah this is a code by Alan Perlis I guess the previous talk here is also I think it really echoes and many of the other talks here language that does not affect the way you think it is not worth knowing with that parting thought it is TCO is part of the scheme spec itself implementation is usually the CPS usually no your your your source program need not be in tail call position you can write a factorial program in it yeah right scheme itself will convert it into tail call position oh no I think I understood your question no is the test based on CPS comparison or no CPS is an implementation detail right it only looks for whether it is tail call or not tail call is you know CPS is one way of implementing I do not I do not think so yeah yeah yeah conceptually it does not I think because continuation of a program respective of evaluation order you have restored the program right you are just capturing the rest of the program in a in an identifier k yeah I do not have not done any CPS in say Haskell I guess my rain might have done yeah