 So we have some leftovers from the Arrowless project. Essentially, one demo and one example. And then we move on to reactive programming with the server, looking at a slightly different project. So imagine that you want to visualize an algorithm, something that I am really fond of, because I would like to take all the algorithms that we'll teach you here. And with a little bit of effort, turn them into animation so that in, I don't know, two hours of effort, I can prepare the lecture that is not just pseudo code, but animation in a way that we can play with. Well, I can do that, but it wouldn't take two hours. It probably would take a week to do it right. So it would be nice if I could do something like that, the visualization of bubble sort, which you can see here, and do it in a way that the resulting bubble code, the one that performs the computation and does simultaneously the visualization. Number one looks like bubble sort, so I don't need to change the algorithm too much. And the hooks to the actual visualization are nicely integrated into the algorithm. They are composable in the sense that there are only a few points where I attach those visualization hooks. And number two, they are parametrizable. I can replace the visualization to do something else. So here is how Arrowless do it. This is one of their demos. And here is how they write the code. So this is the bubble sort function. Nothing special here. It receives, as an argument, a list. Well, it's somewhat special. That's really the only word here, but it's a sort of slight modification that I'm happy to live with. From this list, I extract the list itself, the i and the j. Now I compare them. This determines whether I'm done with the algorithm or not. Then from the list, I get one value and the other value. These are the two values at the arrays, which I compare and if necessary, I'll swap them. And then what do I do? The algorithm is not complete, but you should now see the major change we've done to the bubble sort to make it easily visualizable. Can you see the change to the algorithm that we've done? Exactly. So the loops are turned into recursion, which is a strategy you know from 61A. Nothing special here. Except the recursion here is done with the arrow. We're taking a piece of code, which is this function here, and lifting it into what you may remember as a state machine. And on that state machine, we add one loop. And that loop executes this arrow, a piece of code, this function again. And it does it when this function returns this special value repeat, as opposed to the special value done. So let's continue and see when it returns done. So here is another special case. Which one does this one correspond to? So what is the difference between these two cases? Should this be, I don't think so, right? In the bubble sort, you keep one value fixed, and then you're moving the other one, sort of the bottom boundary. There are two loops, right? And there is i is never incremented. Yeah, it might be the case, right? It looks like it needs to do that, OK? So let's look at the entire code and see what we want to see. So what we have done here, this entire function was passed into this call async. And what does the async do? It takes this function, call it f for now, this one here, and it wraps it into what we call an arrow. And now we are going to take this arrow. This is the value that flows in this program through this dot. And on this, we apply this repeat operation. You can visualize it perhaps this way. And this repeat is going to do this, or it will exit. So we have taken this arrow and wrapped it into something I should probably draw this way, or it exits, right? So the outer box is the arrow that contains only the function f wrapped with this package loop. What do you think this 100 does here? What would be a sensible value here? The number of iterations, perhaps upper bound on the number of iterations. It would be the timeout, essentially the frame rate, or how soon you are going to repeat the arrow again. So it is 100 milliseconds. That's why you saw that it did about 10 swaps per second. Well, it did less because not in every iteration, it actually did perform a swap. And the regular bubble sort has two nested loops, right? One nested loop goes through all neighboring elements. Oh, no, actually. How does bubble sort work? Did anybody take 61B here? You are thinking, oh, well, that doesn't mean you know the stuff. So you are comparing a value against some sort of one particular value there, right? Then if it is lower, you flip it. And if the lower one stays fixed, then you're going all the way to this point. So how did we manage to put these two loops into one recursive call? So bubble sort should have two nested loops. We only have one recursive call. Exactly. So this line here, what it tells us, it says, do one more iteration, one or potentially more than one, iteration of inner loop. And what does that do? It increments j, right? Essentially, it does j plus plus. Until when? Until this condition, right? And if this condition is met, you are not going to repeat the inner loop again. Instead, you are going to take one more iteration of the outer loop, which happens here. And this should be minus one, I think, because we are checking for this as the exit condition. So this should be minus, which is probably just my cut and paste error. And here it does one more iteration of the outer loop. So this is not too far from the regular bubble sort, but it would be nice if we could manage to do it better. So considering going a little bit further beyond both arrowlets show us, could we design some syntactic sugar to actually write bubble sort as nested loops and still have the sort of timeout, right? The beauty here is the timeout that we obtain after each iteration so that the human can actually see the progress of the algorithm. So how would we do it? I don't have the answer here. The paper happily stops at this point, which is a pretty good point to get to. We have a bubble sort that is recursive. The loops are here, but one can read it out. It would be reasonably easy to implement. It does give you a control over how fast the iterations go with the 100 argument. But I would really like to write it with more usual loops so that I can copy the code from a textbook and visualize it that way. So ideas how we would do it. OK, so what we have here, we have one repeat back edge around this arrow, right? But the algorithm has two nested loops, so perhaps we want to have two nested arrows, right? An inner repeat and an outer repeat, right? That would be nice. Perhaps the inner repeat would have the slowdown factor, the 100 milliseconds, and the outer one would have zero just to make another transition. But maybe then we could take these two for loops and directly turn them into arrows, OK? How could we do it? How did we do it in our 164 language in Lua, right? We don't have a direct support for for loops in the language, right? We take a for loop and translate it into what? We just desugar it into a while loop, right? And the while loop is again nothing but a recursive call into that function, right? So could we just change that sugar such that it changes into code like this? Well, in JavaScript, we cannot. But imagine that we are happy to extend our 164 language with such sugar so that what we obtain are loops, not the way we have them today, but they are loops that are built on top of arrows and we have control over when actually the next iteration fires. So would that be possible? So we are really asking now, can we desugar for loop into while loops and then while loops into such iterations that are not recursive calls the way you do them in PA1, I believe, right? But instead, we build arrows of this kind, right? So what steps would we have to take? So we could do it with coroutines. I wonder whether we would even have to do it with coroutines. It would probably make things simpler because the coroutine would be our iterator, right, which goes over presumably the list and iterates over it again, right? The coroutine iterator would be advancing INJ in each iteration. And perhaps that thing, that iterator, would not return immediately, but it would suspend itself by calling essentially repeat. So that might be possible. We would still need to work out details, but essentially you're saying by separating the iterator away from the loop, perhaps the decomposition of that sort will make this easier, OK? One more comment? Uh-huh, OK. So you would have the e-sync arrow have the 100 in it. So you would have a generic run method. OK, so that's probably the most elegant approach. So let me paraphrase what you are saying. Imagine you have a bunch of nested loops, a while loop within a while loop within a while loop. The while would be translated to what? The while would not actually execute. It would return an arrow, like this. Where is the arrow here? It starts from the function, then the function is turned into an arrow by calling a-sync on it, right? So this lambda with the body after you call a-sync, this returns an object that is an arrow, OK? If you do run, it actually executes. But if you first do repeat, and then run, it now runs it repeatedly as a loop. And you are saying, if we have nested loops, let's evaluate a loop in such a way that it doesn't actually run. It returns an arrow. The next arrow above it will compose with the inner one by somehow wrapping a repeat around it. And this repeats, and the topmost code actually runs it. So that seems like the right approach to actually have loops rather than having to deal with this nasty recursion. OK. So this is all I wanted to do is discuss this one cool example of arrowlets. And now we want to move on to another system for asynchronous programming in the browser or other clients that contains a synchrony due to GUI. And that's the Rx system from Microsoft. Why we are switching into a different one, that this one is better suited for accommodating the other kind of asynchronous. So far, we talked about a synchrony that is due to the user clicking at unpredictable times and the GUI stopping. But now we will also introduce a synchrony due to the communication with the server, which introduces a new bunch of problems. So what is Rx? It's a Microsoft DSL for reactive programming. It's implemented as a JavaScript library. But so it's embedded in DSL, just like jQuery. And there is also a bigger version in .NET called Rx, which actually is. Sort of the bigger sibling of Rx.js. And it's statically typed the way C sharp is. The author is Eric Meyer and his team. Eric is the coolest geek you would ever meet. He never wears anything else but a tie-dye t-shirt. If he needs to wear a shirt, he has a tie-dye shirt. And in his honor, we created two t-shirts for this class on a tie-dye setting. And he got one and I kept the other one. And so this is his recent post from, I believe, this morning on Facebook. And it's not complete yet. And then it continues and says that actually, of course, this is all hooked up to the browser via Rx. So Rx is actually pretty cool. Now there is a second generation out. It ships with Windows 7 phone. So all the interactivity you see there might have been actually created in Rx. But what we'll do today we'll use as a running example the sort of auto-completion that you see in Google. You type a fraction of the word and it suggests completions, but we'll do it for Wikipedia. So you type in a fragment of the word and after a while, your client, the browser, decides to send that fragment of the word, a prefix to Wikipedia, get the list of completions displayed on. In the meantime, of course, you could have typed more of the word and so new suggestions come in replacing those that you have in the suggestion window. So this will be our running example. This is based on a nice hands-on lab built by the Rx team, which is your reading, but essentially we are going through it now so your reading should be easier once we are done with this lecture. So first of all, something that the Rx team always emphasizes is interesting to mention. It is this push-pull duality. So when we built iterators in 164, those modeled after Lua, those lazy ones that could go through all permutations of an array and return them one by one, did we build a push model or a pull model? So we build a pull model and who is pulling the data? So the loop, actually, the control in the loop was calling the iterator and pulling the next data from the iterator. The iterator in the meantime would be waiting on the yield statement. So the master was actually the loop and it was pulling the data from the iterator. But of course, there are situations where this is not what you want. Imagine that you are working over data that is just not here yet. It may arrive in a second. It may arrive tomorrow. And you would like to be notified that the data arrives. You would like the data to be pushed to you. Yet you would still like to have a code that makes it look like the data is all there when you are executing it, just iterating over it. Even though in reality, the data will arrive asynchronously later than ever available. So you'd like to have the opposite, the pull model, where you can think of it as iterating over all permutations, but the permutations arrive from the server perhaps at unspecified later time, perhaps never. So this will be the opposite, the pull model. In the .NET lingo, there is an ocean of iterable, which is a data structure over which you can iterate in the pull model, like we did with our coroutine-based iterators. And the dual is the observable, which is an element that you can observe. And when it has another data element ready for you, it will push it towards you. So the typical calls to a web server would be a call to an observable. Sort of the request to the server gives you an observable, which is an element that could provide data later, because the server could decide to give you the entire answer right now, could give you fragments of the answer later. So the answers from a web server are sort of the typical push model. At least you would like to write it that way, without having to pull it from the web server. So these are dual, and it really depends who is the master and who is the slave. So let's look at the basics first, which is we need to look at what is observable and what is the observer. So the observable is the data source. There will be also data consumer, data sync. And you create it like that. So Rx is the library. Inside the library, there is a class called observable, or prototype called observable, since it's in JavaScript. And by calling the return method, we create an observable, so data source, that contains nothing but a single value in this case, 42. Now we create the data sync, the listener or the observer, by calling the observer prototype, and we'll just create it. So this is like new, if you will. And what is inside here? Here is the body. You can think it's a handler that actually consumes the data, and in this case just writes it out at the end of the document. So if this is our program, what will this program do? When you run it. Is it going to do anything? Is it going to print 42, wait for 42? What's the guess? Exactly, it's not going to do anything, because if you look at these two fragments of the program, they are not connected. Answer is a variable that contains a pointer to the observable, the data source, and that's not hooked up to the observer at all. So we somehow need to tie them together with an edge, sort of a data flow edge, that will carry those values. How do we do it? Well, here is how they do it in Rx. You essentially subscribe the observer to the data source. And when the data source has a new value, it will push it to the subscriber. So you have essentially built now a data flow program where this has only one value 42, and you send it to this observer, which in this case only prints it. Of course, this example does not motivate Rx, but it explains first two key concepts. So that is clear, I hope. So here is how it would look like actually in the entire web page. So here we have the script tag, which closes here. Here is the code that we have seen, and here is the button on click, which invokes our main program. But otherwise, this is just the usual JavaScript HTML wrapper here. All right, now the data source, the observable that I showed you so far, was sort of dumb. The only thing it contains is one value, 42. Often you want to actually go over sequences of values. And the values don't need to all arrive at the same time they could come now, later, and so on. And this is actually the more interesting use. And so let's walk over those. So so far, we'll create a variable that will point to our observable. Right now, there is no observable, but I'm going to show you how we can hook up different ones. And let's see how we subscribe to these observers. The call to subscribe has actually three functions. And they are invoked depending on what happens on this sequence, think of it as data stream. This one is called on the next element. This one is called on error. Imagine the connection was disconnected. And this one, when it is all done, but done correctly. We see how what we do here is the piece of jQuery takes the next value, it attaches a piece of HTML to the end of this DOM element, so essentially prints it to the web page. Now here, we create an observable data source. This one is empty. So the program just says, OK. I have read the stream. It's empty. It's comforting that this works. Here is one that explicitly throws an error. It creates a stream that does nothing but throws an exception. And you see it on the receiver side, on the listener. And here is the one that we've seen before, which contains only one value, and then it is done. So this is a single element sequence. Now it's becoming more interesting. Now we are creating a data source that sends three values starting at 5, 5, 6, 7, and then it completes. And this one is synchronous. It sends them right away rather than doing it with a delay. If you want to do it with a delay, here is an analog generated with time. Here is the initial value. Here is the test. So we are going over a few functions we are supplying here. This test is meant to be done. Here is how we increment the value. Here is the value that we actually produce. And here is the timeout. You could say this is the pause between generations. So this just says how we are spitting those values into the stream. And think of it, this is our for loop where this is the body that produces the value. And we have the extra capability of setting the time, which tells you how soon to generate the next value. So this could be used to do the bubble sort version in RX. Now this is all synchronous so far, except for the timer. So now let's look at events like mouse clicks. So here is how you would do it in jQuery. So I let you read the code. It should all be familiar. And I'm underlying the part where it will differ from how it is done in RX. So who can characterize concisely the difference between jQuery and RX in terms of how events are handled? So we are explicitly creating an edge. So that's correct. But there is another interesting thing there, which is the edge goes clearly to this handler. We are subscribing on the receiving side of the edge with a function that is exactly the same here and here. There is no difference in that function. It's just plain jQuery. But what is on the source side of the edge, right? The subscribe place is the edge. It attaches the observer to some data source. What is the data source here? So we have essentially took the document. You could do it with any other element. But we took the document. This is the jQuery value of the document. They modified the jQuery a little bit in RX. And we turned it into what? We turned it into a data source, right? So you have your document here. And now you call toObservable. And what you get on the output is a data source value. So you see, this is not very different from the arrows business. We have now taken a document, a plain DOM tree, and turned it into something that now can be composed, in this case, through the subscribe edge. And now we can do subscribe on it. So this example here does not really motivate RX either, right? In fact, the code is longer. So what did we gain? Clearly, programming languages are all about clarity, composability. So clearly, I have an agenda here. And we are going somewhere. So what are we going to compose in here, or make it easier to compose? So try to remember what is our original goal. The original goal is to write the application where I'm typing the search word. And I want to get, as I'm typing the suggestions from Wikipedia, essentially, the list of Wikipedia pages that start with what I typed so far with this prefix. So think about that application. When would it be useful to compose something into our pipeline, right? We have built a pipeline here. From the document, we created the data source. Now we subscribe onto it this printer function. So whenever something flows down that pipeline, the printer prints it. In this case, whenever I move the mouse, an event occurs, and the printer prints the position of the mouse. What might I plausibly compose here inside the query? I want to snip it and insert some other filters, if you will. We could also compose sources. So that's not something we are going to do. But the, uh-huh, uh-huh. Oh, OK, in that sense, right, exactly. Fantastic. So this is one example that I have. One of my data sources will be the, not the mouse, but the entry, the text field where I'm typing the word. That's one data source that provides these partially typed strings, OK, those terms. And then I somehow need to hook it up with the other data source, which is the Wikipedia server. So that I will have to do. And yes, indeed, I will insert it over there. But before we get there, there are other more mundane things that I will need to insert right there. So imagine you are typing really fast. Do you really want to send to the server a new request after each keystroke? The way I'm typing, you could, but the way you are typing, probably not. So you type faster. So what would we do to somehow reduce the load on the server? So they call this operation calming. You're looking at the stream of values, and they may change quite a bit, perhaps because you typed five characters in a second. They might have included arrows and delete characters and so on. You would like to calm it, and only after some time you say, oh, I see the user stopped typing for half a second. Only then you send it to the server, OK? So that would be one thing that we want to perhaps insert even there into the pipeline before we send it to the server. So before we get there, let's see some simpler things that we will also need, projection and filtering. So here is how you would write projection in the plain JavaScript event handling. So what does filter mean? You receive an event, and you don't want to process every event, only if it satisfies some property, some predicate. So in this case, you look at the x-coordinate and the y-coordinate of the event, and if they are the same, only then you actually handle the event. So you only handle events that happen on this diagonal, x and y, well, from your point of view this way. You may want to project it, which is, well, you have an event which has a target, and the target has a value. If it is the text form, the value is the string itself, the term that you are typing. The target has many other values, but you only care about this value. You may want to extract it. It's called projection. And send down the stream only that projected value because you don't care about timestamp, color, and 500 other things, OK? So this is how you would do it in plain JavaScript. Let's see how you do filtering and projection. Somebody just won a jackpot. You know, we'll tax you. So in Rx, the way you would do it is you do the thing that we have seen already. Take the document. Now you turn it into observable, OK? It's here. And now you apply a select operation, OK? Which is the projection operator? It receives the event, but what do you return? You return this hash map, which has two fields. And this one has page x, page y, but all the other fields in event, and there are many others, are completely ignored. So you have projected the event into this struct that contains page x, page y, ignoring everything else. And now as a result of this composition, that composition, and this projection, you have now built a stream that looks like this document. You turn it into a stream. Now you have a select, which uses this function. And now what we have traveling down the stream is only page x, say 7, and page y, say 8. And now you can compose other things over here. And that's it. So here we are projecting the value of that stream, again ignoring everything else. Once we have that, this is after it's all filtered, we can subscribe to it, attach something at the end. This is what does the work. So notice the difference between select and subscribe. Select essentially inserts something into the pipeline and keeps sending the values down. The function here returns something. It is sort of a filter that processes the data. And here we have subscribed, which you attach at the end. You see it doesn't return anything. It has, as a side effect, made a change to the DOM. And that's it. Is that OK? All right. So filtering is where, see what's selected. Select is like a map. A value comes in. You apply a function, and it's sending down the stream. Filter is where you decide that some values should be just dropped. And that's done with the where operator. And here is the condition, which decides whether that particular value is passed on or dropped. So you can now take the moves, which we have already projected away. And now we can look at just the x field and y field. And now we attach to at the end the subscriber, which again prints it like before. If it is straightforward, it's because it's designed so nice that this filtering is straightforward. And if you do many filters, you can easily read them because they're all in one sequence. All right. So this is so far just the basics. We'll get to more interesting stuff shortly. And what we've seen that the streams are first class citizens, which means they are values, which you can store in other hash tables. You could even take streams and pass them down other streams. Might be useful. And because they are first class values, you can compose them with composition operators. This is not really different than with the arrow stuff, in arrowlets. OK. So now we are going to manipulate sequences. And now we are ready to do your calming. And the motivation for it here is that, well, before we get to calming, we do a distinction. Imagine that this is where you are. Imagine this is what you have typed. Your cursor is here. Now with shift left arrow, you are going to highlight T. Now you type T, and you put the cursor back into original position, and you have typed T into T. You have changed T into T, which means you haven't changed it. Here is the sequence that you will see. So if you look at plain events, of course, changes have happened to the value because events have been created. You did a keystroke. But the value is the same. So it would be nice to have an operator on our data flow sequence, which completely filters out these sort of editing changes, which make no change to the actual value that you care about. And the way you do it is that, well, you define an operator that you again insert into the stream. They call it distinct until changed. So look what we've done here. They have these nice syntax that you can write the entire pipeline under each other, make it easier to read. You take another document. You just take the text box, which is the input field, turn it into an observable, a data source. Now we do a projection, which you just take the value from that text field. And here you suppress all those that are the same as those before. So if you have down the pipeline flowing the values A, and then later value AB comes in, and then you have AB, then which of the three values is going to be suppressed? First one, second one, third one. The third one will be suppressed, right? Because it is not distinct from this one here. But if the third one is A, again, it will not be suppressed by this operation. OK? Now you can do throttle. Throttle is the calming operator. They call it throttle. And what it does? Well, you set it here. And throttle is an operator that sits in the pipeline and waits for 1,000 milliseconds. And only then it says, OK, if nothing has happened over the last 1,000 milliseconds, then I'm ready to send the value down. If new messages, new events arrived in the last 1,000 milliseconds, then clearly things are still happening. And throttle will wait. So throttle waits 1,000 milliseconds after the last event, and then it says, OK, it's time to send it down. What it does with the rest of the events? It just drops them on the floor. So it does behave the way you want. You see how easy it is to incorporate it into the program. You don't need to have another handler. You don't need to set up another interrupt. You don't need to compare the values. You just insert this into the pipeline, and that's it. This is why frameworks like RX are getting popular. So the order does definitely matter. OK, now let's see. The question is, would it make sense to reorder these two? OK, I see the problem. All right. So presumably you have in mind a situation that could show that this is a bug, potentially, right? So let's see. I'm guessing this is what we have in mind, right? I will type A, then I will type AB, then I will type A again. And I can time those events in such a way, right? And imagine that these are flowing down through this part of the pipeline, right, before throttle. And I could do it in such a way that throttle suppresses AB, presumably, right? And then distinct until change, you'll only see two As. And then as a result here, I would only see this one A. But that seems OK, right? Because presumably I still want my list of completions to see the completions for A. Because I have typed A, the system showed the completions for A. I did AB, but then I deleted back to A. So it's correct to still see the completion for A. By the way, the right reasoning, the system by itself, it's high level. The programs are easier to read. You still can make bugs in it, and this might be one. I think if you just move the cursor, well, now the question is whether the data source keeps sending the values down even after you move the cursor. So let's assume that it does. So I move the cursor each time the value, the same value, is still sent down. Clearly, distinct until change is going to suppress them and only use the first one. And that's OK. You still have the correct completion. So that works correctly, I think. So can somebody think of a sequence of keystrokes, including the timing? Because timing now matters. That somehow will lead to an incorrect situation. So what is the invariant that we want to maintain? We want to maintain the invariant that whatever is typed in my text window, eventually, so after some calving period, it's also going to have its completion in this window of hints. If you could get this program into a state where there is some text in the window, but the completion is for some other string, then we have a bug. So it's clearly difficult to reason about this, and indeed, asynchronous systems are quite difficult to debug. So rather than thinking, since that's painful and slow, how could we automatically detect whether there is an error? Yeah, I think that logically, it seems to be lost just before the server is suppressed. And now it's interesting to think about it, if you want to think about it, whether indeed these two operators in this particular operations commute. Can you reorder them and show that indeed the behavior would be the same? So do they have the commutativity property that I can swap them and the program will behave the same? No, yes? They will probably both maintain the invariant, I said. So in that sense, the program will be correct, but they might behave differently. So this is another thing that we are not talking much in this course, except for the static analysis of programs in prologues, where we were detecting how values flow through the program. Aside from that, we are not talking much about automatic reasoning about programs, convincing ourselves that they are correct, finding bugs in them. That's getting more and more important, partly because of asynchronous and distributed computations, which are difficult to reason about, as you just saw. This is clearly a hello world example, and it's already not clear for us to say that they're swapping these two calls will cause an undesirable behavior, because timing is involved. OK, so talk about throttle. Now, two more operations. You can insert into the pipeline a do node, and the do node does what we expect. The do is taking the value that is flowing by and applies a function on it. And that function in this case does what? Essentially, again, prints it using jQuery into the DOM. It extracts some value from what's flowing by, the value and the timestamp, and it prints them. How is do different from select? We have seen select, which also took the value and sent something down. So can you recall what the difference between select and do? Uh-huh, OK. Exactly. So do is sort of just a peak at the value. It sort of builds a T, takes the value, listens on it, or a tap, does something with it. In this case, it has a side effect. It prints it, whereas select would manipulate the value, usually project something. It could wrap it as well, and it sends a new value in. So select is a map, and this is something like, I don't know, leak it out, print it. And there is also timestamp, which has a corresponding operator. Remove timestamp. And what it does, adds a timestamp to this message, and then removes it. So it's one of them is a wrapper, the other one is a remover of the wrapper. All right, so now the real fun starts. We are going to deal with the server. Can you imagine? Yes, please. OK, how does do differ from subscribe? So can somebody try to answer? How does do differ from subscribe? So subscribe doesn't return the stream. So it truly terminates it, and do still returns a stream which you can then listen on. I think it would still work just fine. So perhaps that's a good point. Perhaps it is a question of cleanliness, that you should not use do if you are really subscribing and listening to it. Good observation. OK, so what can happen with the server? What are the things we may need to be careful about? OK, it could be that we don't get a response, so let's actually even not deal with that since that's too hard. Yes, but the failure that the message doesn't come in is difficult, because we could deal with it explicitly with the timeout and the requesting it again. But that is something we are not going to deal with. In addition to that, what could happen? The messages could be out of order. Again, a fantastic observer, I guess. The observable observer works really well. What else could happen, you think? What if the response itself doesn't? The response could come in chunks, not in the same time. So the response does not need to be atomic in the sense that its pieces could arrive separately. Imagine that you are subscribing to a stock update. Presumably each time the price of the stock changes, you will get the next thing. You would like to model it as subscribing to some stream to the server that delivers pieces. So these are essentially the same thing. You lose the message. We are not going to deal with that. There is out of orderness. And then the response is come back later. All right, so here is what we want to build. Here I'm typing the word, and here is the completion that I get from the server. Presumably the server, Wikipedia, contains an API, meaning you send it the right URL with the search term, and it sends you back the list of completions. It's as simple as that, OK? So here is how you will do it in JavaScript. There is this call Ajax, right? So the dollar is the JavaScript variable which contains a method Ajax. And the method Ajax just builds one huge table, which contains the URL. In this case, this is the API, written in PHP, obviously. You say, I want to send you the data in the JSON format. Here is the data itself. Here is the search term. That's the most important one. But there are some other parameters that we are sending to the Wikipedia search engine. And then here are the two handlers. This one called onSuccess, and this one called onFailer. And what do they do? Well, imagine that you get successful data. You have some ID in the DOM, a place where you are printing these completions. This would be this here. This window here is the node called result. So you're going to empty it. And now for each data in here, you are going to append the result. The result is here, to that box. So have a look at it, right? Because you need to understand what is going on here. So the first thing you want to know is that this here is a string, or not a string. Is it a string? What would make sense for data sub 1 to be? We are taking the first field sub 1 because the struct that Wikipedia sends you has something in sub 0. I don't know what. But sub 1 has the actual payload. Exactly, because we are calling dot each on it. Well, we are not quite calling it on it, but we are asking dot each to iterate over it. So this is an array of strings, right? I'll use this type notation. It's an array of strings. And what is that array of strings? These strings are each one possible completion. Now, here is how it will be done in Rx. What do you think this will do? It creates an observable, a data source attached to what? What is the source here, actually? The server, OK? And the rest is pretty much the same. Same URL, the keyword, and so on. Why change it, right? You want to keep the same stuff that doesn't need to change the same. Now, OK. So let's see, what is the format of this? So if we think of, let's imagine I, OK, let's change this a little bit. So this returns an observable, but let's make it parametric. Now, compare these two. What we have here, it takes this description of the request to the server with a particular search term. Here, we're assuming I have typed react. And it returns you this observable to which you can then attach listeners, filters, and so on. Now, see how we are parametrizing it. Here is the same call, except now I'm parametrizing the search term. So I'm just defining a plain function, search Wikipedia, into which I pass the search string. And the result is what? An observable on which I do a projection. Why? Because I don't want to deal outside the function with the fact that the data itself is in the first field. So I project that array of strings out. And now I have a question for you. So what is the type of search Wikipedia? The function clearly receives an argument of some type and returns a value of some type. So a warm-up question, what is the input type of this function, search Wikipedia? It's a string. Well, or is it actually a sequence of strings? Is it one of those pipes that carry strings? No? No, it's just a plain string. That's correct. It is a plain string. It is not a pipe that is shipping strings down that pipe. What is the return value? It is an observable. It is a data source, in other words. But observables have some types. Imagine we want to describe what's flowing down this observable. When we attach ourselves to the observable, we want to know what's in it. If this was, in other words, in .NET and C-sharp, then we would need to spell out this type. This is JavaScript, so we don't need to spell it out. We don't need to enter the type into the signature. But we better know what that type is. It would be array of strings. OK. Let's see if all of us like it. So is it an array of strings? Clearly, previously, if you go here, we have agreed that data1, which is the actual payload, the result of interest, is an array of strings because each iterates over them and prints them into this result window. So presumably, this is also an array of strings. After all, we obtained this observable here at this dot. And now we are projecting away everything. We are projecting away the actual payload. And we know that this is the array of strings. So clearly, clearly would mean I can clear it, but I cannot clear it. All right. This is an observable that has arrays of strings. But there is now something strange about this, right? All this strange. Well, I don't know. So but really, the high level point is what? If we have a stream and if we say that it's an observable of ints, do we know how many ints are going to flow down that observable, that pipe? Could be 1, could be 2, could be 0, right? So in other words, the types that we are using here to describe our observables do not specify how many of those values, in this case, arrays of strings are going to come down that pipe, right? This type observable of arrays of strings could legally describe a pipe that sends nothing and just closes that connection. It would legally describe a pipe in which you have one array of strings and then later another and then later another, right? The type just cannot distinguish between them. Well, this is what I'm trying to say that, OK, let's see if I can use a piece of, OK, let's do it here. So imagine you have the following sequence. I'll use these brackets, these funny brackets to denote the sequence. What is the type of that? You will agree that this is observable of int, right? If I give you such a sequence, it's also of that type. Well, the way to think of it, it is somewhat confusing. No, no, think of these brackets here, these brackets here as this observable is parameterized by a particular type. So think of Java generics, right? Or C-sharp values. If I create an array that stores only integers, I would write it as array of int. What does it mean that the value of the type is an array that stores ints? But notice I'm not saying how many elements it has. Could be an array of length 0, which has no integers. Could be a, OK, you need to speak up because there is noise in the background. Oh, I see. So, OK, this is what you miss. So let me, I use this as a special syntax. OK, I see now how it confused you. Let me use, what syntax do you want to use for sequences? Would that work? OK, or we could say, I know what we'll use. We'll use, we'll use this as a pipe. I like this syntax. I hope your parsers will support it, this syntax. OK, so the first sequence will send you three integers. And it is of type observable of ints, pipe of ints. The other one sends you one. The type cannot distinguish. The type just doesn't describe that property, the length. It is the same as with lists, right? And with arrays in normal languages. Sequences are like that, except there is this time question there. And so if you look at that, we are saying, this function, we'll take the search term and return you this pipe. Think of it as a fire hose on which you can then listen. And down that fire hose, eventually, might be dangerous, yes, bad metaphor, eventually arrive an array of strings. So how different is it if I told you that? So yes, please. OK, so that's a good question. This is related to how Wikipedia designed its API. Wikipedia could decide that instead of sending you all the completions at once in one array, Wikipedia could establish a connection with you and feed you those completions one after another. But they know the results right away, so they send you all completions in one piece atomically in one array. Yes, if you took them too long to find them all, then it makes sense to serve you those completions one by one. You would perhaps sort them on the client, display them. But the decision was that they send them to you all at once. And they don't send you anything else later. That's at least the agreement on that API. But this is why it is an array of strings, because they package everything into an array. All these completions send it to you once. We know from knowing more than the types know that there is not going to be another array down the pipe, only one, but the type cannot tell you more. It just says that what you read from that pipe is going to be an array of strings. OK, excellent question. I love your question, guys. So why do we need an observable? They send them all at once. And it seems like, OK, they are sending them all at once rather than one after another with some delays between them, with just one value. Why do we need an observable? I want everybody to think about it, because this is crucial. The beginning stuff, the throttle was piece of cake. But this is somewhat important for composition. All right, so think of a different kind of server. Think of a server that gives you the current time of day. So you know it's only one value. It's not going to be an array of times. Would you want an observable to such a server? Or would you say, well, it's only giving me time, so I'll just block and wait? You would not block, right? Because it may take two seconds for the server to tell you the time, and that would freeze your user interface. So it's not a matter of how many you are sending down, but it's a matter that the response, whether it's one value or more, will take a while and you don't want to block. You want to subscribe and then wake up when it arrives so that the user can keep typing and growing that prefix. Excellent questions, by the way. It's not at all obvious. In fact, even if no value is coming down, just closing that connection, remember, there is this completion versus exception. Even then, you may want to have observable so that you can do other things while it's happening. Now, you could imagine Wikipedia eventually growing that API and telling you that, oh, I will send you another array of strings later, assuming you keep that connection open. What would potentially come with this additional array of strings? So they could give you top 10 right away and then those further later. Or they could give you all of them, but then people add Wikipedia pages. They remove them. They could send you an update later, right? There are many reasons why you may want to have an update, but this server is simple, yeah? OK, so it is an array of strings, so fantastic. So this function, search Wikipedia, gets a single string, plain string, no pipe, and gets you back a pipe. I think this is probably the hardest slide to understand. Well, maybe not. Now, so what do we have? We call this function with our term. It's hardcoded here. This function points to that pipe, right? So it is a pointer to this pipe connected to Wikipedia. And sometime later, the array of strings will arrive, OK? Now we can subscribe on it and guess what we do. Well, we just iterate over the results and print them into this window. Now everything is simple, except we haven't dealt with this out of order stuff at all. So now let's put everything together, all these pieces, and deal with the out of orderness. So now we look at the complete app. So here is the piece of HTML so that you have it. And here is the input box called search input where we are actually typing. Here is the DOM element results into which we are printing those results. And here is another DOM element where we are printing the error. So there is nothing but the web version of printf. Here is that search Wikipedia function again. No change to what we had before. Now the input here is throttled. So we have a document. Already, only when it is all built, we are going to attach this, right? So we attach to the search box. What? We turn the search box into a data source, which on key up, when you release the key, will send you that value. Here you project it. You just extract the value. So at this point, we have these values flowing down. You throttle them with a quarter second delay. We'll compose other stuff down there. And so what do we have? We now have the input box. It's here. Here we have those terms flowing down. This is the pipe of terms. Each time you type, a new term shows up. Now we need to attach the Wikipedia service to it. And the results are coming back. Seems simple. So let's do the composition. This is something that we have discussed. Yes, I think we are ready to ask the question. So here is where things become, again, somewhat tricky. So what is terms? Do you remember what is terms? What does that contain? Let's go back and see how we built terms. So we have built terms. So look at the code. Should be, by now, easy to read. So what does terms contain? Is it a single string, pipe of strings, some other stuff? Exactly, right? So each time I do key up, a value is sent down. We extract the value. Ignore everything else. We throttle it here. You could add distinct until stuff here. So indeed, this here is observable of strings, right? Terms are observable of strings. Everybody agree? All right. So now, for each string flowing down this pipeline here, we want to make a query to Wikipedia, obviously, right? What will be the type on the output? What will this pipeline return at this point? Is it observable of arrays of strings? So I'm trying to figure out what to put here. So I want people to think about it, right? This is crucial. Imagine I have here observable of ints. And I put here a function f. So I here do a select. Remember, select is this projection, which is essentially a map operation. And imagine that the type of f is int to string. What would be the type here on the output? So if f is a type from ints to strings, and I have here observables that contain ints inside, what do I get on the other side of the select? Select is like a map. So obviously, what do I get? Observable of strings, right? It's as simple as that. You could say you lifted that function from ints to strings to observable of strings to observables of strings. So what is the right type here? If I put here select, what do I get here on the output? Exactly. We get an observable of observables of arrays of strings. So now, if we use select, then what we get back is a pipe that carries in itself those pipes, which would be fine. But this is probably not what we want, OK? So we don't want select. We want some other operator here. What is that operator? So again, goes for the reasoning, the input is observable strings. Type of this function is string to observable of array of strings, right? We went through this. And therefore, the result of map, or you could say select, is that. It's a pipe of pipes of arrays of strings. But what we want is that, right? We want an observable that contains these arrays. What are these arrays? Now we will have an observable that does indeed contain multiple arrays of strings, right? Those corresponding to different requests to the server. And there is an operation called select many. It does exactly that. It takes those observables of observables and flattens them. And so this is the right operation here. Not select, but select many. And all you need to know that it flattens it, OK? So we are almost done, except there is this little problem here, right? Imagine that we have input and I type react. Now I type reactivate. I make one request to Wikipedia. Then I make another one on that more current word. But the first one takes longer. So I display reactivate. And then the strangler arrives and overrides my output. With a value that is out of date. It's no longer in the string. This is the out of order problem. So we could build essentially an analog of TCP, keeping track of timestamps and dropping those that are later. And indeed, this is what RX does under the covers. But we don't do it ourselves, right? So what we do is we want to kill those that are old. So as soon as we issue the new one, we want to get rid of the old one because we are not interested anymore. And the way you do it is that you do a select. So now here, indeed, we have observable of observables, but we have a switch. And switch does exactly that. It has the effect of killing those that are older. They are in flight. They still could come back to your client, but switch is going to suppress them. So you can think of a switch as something that drops out those that have been already absoluted. So this is essentially a picture of switch. And what it does, it takes these outputs that are here. And it sort of nicely streamlines them into the new observable. So that's it. I think we finished exactly on time. And so we have now not only ways of composing these streams, but also dealing with out-of-order operations should you care. You don't always care about getting rid of older things. Sometimes being out of order is fine, and you may indeed want it for performance reasons. So now you have this nice control over dealing with this. So that's it for today. Oh, one quick announcement. I will be away next week. So what I will do, I will record the lectures and you can then watch them. I'm afraid you cannot ask me questions that way, but that's what we can do. Otherwise life goes on as normal. Good luck finishing those design documents.