 Hello, everyone. This time we have Yuriy Kremzer, senior software engineer at Retro.net hat, who works on the Haukilo project, talking to us something about reactive programming. Welcome. Hello, everyone. Can you hear me? Okay, so today I will be talking about reactive programming, and the subtitle of my talk is, Async can be easy. So before I tackle the reactive programming, let's look what the Async actually is. What is asynchronous? So is this Async? No, no. That's Async. That's something different. I like this definition that Async is multiple events and computations happening simultaneously. That doesn't necessarily mean that there should be some multi-training behind it. We are not going to talk about parallel programming and multi-training, nothing like that here. Because imagine JavaScript, there is just one event loop, and you can still do kind of parallel things, which are kind of virtualized. So what asynchronous programming is as well, it is annoying. It's kind of hard to reason about the programs that are doing multiple things at the same time. Because you don't read them from the top to the bottom. There are places that are happening that they call you. It's kind of not like a book that you read from left to right and from top to bottom. It's more difficult. And it's also difficult that each language, each framework has its own way how to deal with the asynchronous events or asynchronous calls. I think the most well-known thing is the callbacks. It's basically that you will provide a parameter to do a function that will be called after function completes. But there are also things like promises or futures called in Java, listeners and other stuff. What are these concepts have in common that each concept covers only the part of the story? There is nothing more general than that. So let's look into the particular primitives. Callback, as I said, it's a parameter to a function. So if you call the getData function, it doesn't block the computation. The computation continues. And once it is done, the callMeBaby callback is called. The different approach is to use futures. You probably know that. You can, again, call getData method. And you will get some kind of handler to the future value that is not immediately there. But you have to wait for it. There are multiple implementations of futures. For instance, you can call get method on a future that will actually do the blocking call and you will wait in the program. Oh, yeah. And today the situation is that the microservice architecture is quite common. So it's not uncommon to end up with this state. Imagine you have a single REST API in which you call multiple REST APIs, different one. So you can have future of list of futures because there are multiple calls. And, yeah, of course, you can have the first version, the future of getData, but in the last case, later case, you actually say that you return the intermediate calls directly to the user and user can handle them on its own way. You don't have to wait for all the calls to finish and then return the composed future. So how to deal with it? We've got observable. It's a concept from reactive programming that will tackle those problems. So where to find observables? You can use them in almost every language and there is RX libraries for them. So it's extension to languages. I like this quote. Telling a program there is library, it's like telling a songwriter there is a song about love so everybody wants to write the library itself. So is this a situation what I'm talking about? You probably know this comic, but just read it. Let's hope it's not the case. So what is functional reactive programming? Here, there is a word functional, but my talk is called reactive programming. Why functional? It's because the reactive programming is quite general concept. Basically, each spreadsheet does the reactive programming because imagine Excel, you've got the cell and then you've got the dependent cell and if you change the first cell, it will propagate the changes to all those dependent cells and that basically the definition of reactive programming, the propagation of change and data flows and things like that. So here we'll be talking about functional reactive programming and nowadays it's basically represented by those three points that there is something called observer and observable. So you can imagine observer as a guy who observes event on observable. Actually observable is pushing the events to observer. It's calling the events on observer. Observer is also called a subscriber. There is also a lot of combinators. Those combinators can be called on the observable collection and you can do pretty useful stuff with it. You probably know those operations on collections that you can filter the data, you can map data, reduce data and things like that. It's basically relational algebra. But there are also combinators that deal with time aspects. We will look into more detail later. And the first aspect is how, when and where things are happening. These are often represented by schedulers. We are not going to talk about this because it's kind of implementation detail and it should be an interactively talk about reactive programming. So why the functional, I basically covered that, but because it's cool to do a functional way. And there is a lot of economic background behind it. Another reason is that it is good to know that the future observable collection optional and others, they are all moments. It means that if you know that, you can work with them in a kind of same way. There is also, every time there is an operator in mathematical theory called bind, but often it is flat map in reality. And you can map those observables, sorry, those monads to, you can combine them with another run and still stay in the monad. So for instance, it's heavily used in the future monad. In JavaScript, you can call then on a future. You call a future and then you can chain those future together with the word then. So you have one future then, another future then, the third future, and it will end up in the one big future that is basically the chain of the three calls. That's possible because then is actually the flat map. It's just the theory behind it. And if you realize that, you can actually use something similar for optionals. Optional is the object that encapsulates thing that might be there or might not be there. It's basically how we deal with now values. And you can actually flat map the optionals to kind of still stay in the optional state. It's like quantum state that you still don't know if it's there or not. But you can still compute or process in a code later and later. Another reason is that pure functions, unobservables all functions are pure. That means that there is not all, but almost all there are functions that can have side effects. But most of them are pure. That means that if you call them over and over, it returns the same thing. And it is good to deal with side effects what actually observers does. This type is written in pseudo-scala syntax. And it's basically what the getData method should look like because it can throw an exception. That's what the try does. The data might be there or might not be there. So that's why the optional is there. And it's not there immediately. So that's why the future is there. Let's look to the landscape of the programming data structures or patterns that are used for async programming. I like this slide because it's, I haven't looked, I haven't thought about it's problem in this way before I know the reactive programming. On the x-axis you have the quantity if you are dealing with single value or multiple values. And on the y-axis you've got if it is a pull model or push model or if it is a synchronous or asynchronous world. So objects, just simple stuff. You can call method on it. If you have more objects you basically end up with iterable or collection or some array in other languages. And if you want the one thing but you don't have it right now you can use the future or promise. I think it's called task in Python. But if you have more things than one and you've got, you don't know, you don't have them right now the observable is the right thing too. So let's look into the observable. So what is observable? It's represent push-based collection. It's not very useful definition. It can be combined with other observables. It doesn't say much higher. It's monad. We tackle that. I like this definition. It's any number of values over any amount of time. It's quite intuitive. And I like this thing. So you can imagine as a stream of events, some data. So once again, difference between iterable and observable. The iterable, it is the consumer who calls the methods on the collection. If the consumer wants a next value he calls the next on the collection and he negates the value. It can throw us an exception and then it can return if there is nothing. It's normal blocking stuff. On the other hand, you've got observable where the producer actually calls the method on consumer. So these three methods are quite important because that's how the observable communicates with the observer. And it's how he says to observer, I've got new value. So if I've got new event of new value, I'll call on next on observer or on subscriber. If I've got error, I'll call on error. If there is no more elements in the stream, I will call on completed and the stream is kind of finished. And there is a grammar that can be used for kind of all the streams. So there can be an arbitrary number of events. That's arbitrary. Many times you can call on next method. And then you can or don't have to call on error or on completed. So these are kind of happy paths. There can be infinite stream. There can be stream with three members and then on completed or there can be stream with four members and then there is error. It is kind of memory effective because what is happening in the collections, you probably know those operators like map, filter, and you know, reduce and zip. These are well-known functions having all languages. But normally if you call the map, for instance, it will create an intermediate collection which needs to be garbage collected depending on the language. It is garbage collector or not. But in observable, it doesn't create anything, some intermediate results. It just sets the plumps and all the plumbing. And then if some subscribes to the observable, the things start happening. So the first piece of code, if you run this, it actually does the stuff. But the second, it actually set up infrastructure and does nothing. The observer have to subscribe and then things will start happening. And they are happening in a way that it is kind of hot potato processing because you have an event and you apply the flat map function on it and map function on it and filter function on it. It is not like you iterate through all the events because it is not even possible because the stream can be infinite. It is like one by one processing. So how can we create observables? There are factories methods for it. So, for instance, the just method is pretty simple. You just provide a number of parameters into that method and you will end up with observables with those elements. But there is no time expected. They happen at the same time immediately. There are factories for dealing with time like interval or timer. For instance, interval, you provide a time period and time unit and it will create a new number after each time period. It doesn't look very useful, but you can combine those observables to a more complex one so it can make sense with composing with another one. As I said, there are many, many combinators on observable. Here I explicitly say the type of the map and flat map because the flat map, as I said, is kind of a good operator in this functional world. It is because, you know, you probably know the map. It is the simple function with the individual except a function and it is called a collection or observable and it applies the function on all the members of the observable. But what the flat map does is it is more stronger because it is calling a map and then flattened the structure. So the flat map takes the function which is going from the type of the events to another observable. So we are kind of nesting into, if we would call the same function on a map, not flat map, we would end up with observable of r. But we don't want that. We just want to flatten the thing to be able to just call the next method on observable. You know, it is a flow API. You want to call a method on it. You kind of massage the content and you want to call the next function on it like filter. So you don't want to be nesting deeper and deeper. Yeah, there are more of them. Zip, merge, concat, group, buy sample. For instance, the sample is lossy compression. You can have multiple events and the sample actually takes another stream and it creates a new stream which will have events only on those positions where the parameter of the sample have some data. So the parameter of sample is completely ignored. It serves only for kind of signaling that here take the data from the first stream, here take the data from the first stream and so on. I will have demo so we will demonstrate the sample method at the end. Yeah, so this is something called marble diagram. It is a thing that visualizes actually the operators. It's much better than describing it by words because you probably know that it's difficult. I probably didn't explain the sample operator well. So let's look into the sample operator, for instance, if it is there. Okay, it's not there. So this is the stream on which we call the sample and this is the parameter of the sample method. So it ignores the events from the second stream and takes the events from the first stream on the positions where there are some data in the second stream. It is nice that you can change the order of the input and see the result. Okay, so there is something called Holt versus Holt distinction in observables. By default, all the observables are Holt. This means that they are lazy that if no one subscribes, they don't do anything. After the first subscription, they will do the stuff, they will process the events and so on. On the other hand, we've got Holt observables which are kind of cues or something like broadcasting stuff and there doesn't care about subscribers. They just produce data and you can subscribe on them and you actually miss the data. You can subscribe in the middle of the stream while the first time, every time you subscribe, you actually replay all the events for you. The frame will replace all the events for you. So it's something like this smart thing. If you don't subscribe to colopserable, nothing happens. Okay, so let's look at some code. All right, I've got simple demo that will, oops, that you demonstrate how observables can be useful. So here, this is visible. I think it is more than that. So I've got a URL and I will be calling get functions to stocks. I've got also HTTP client. Okay, HTTP client. So this is the guy who will do the get calls. So I've got fetch method that is actually doing the blocking call. So this dot execute is a blocking call. It does the get request and waits until the response goes back to the client and then it processes the response. So it will pass the JSON and take the value of the stock that is provided as a parameter as a code. Nothing fancy here. I've got second. So let's run the code. Here is the simulation. So I'm creating an observable by dot create a method that is a factory method that actually creates the observable. So it takes a parameter as a, it takes a subscriber as a parameter because as you know, the observer is calling stuff on subscribers. So it needs the subscriber on which it can call the on next or error or completed method. And I'm just using the infinite loop that from these codes it will map them to a stock fetcher fetch. So instead of each particular string there will be actually the stock value instance. I haven't, there is also stock value, but this is just a code and a value of the stock. And for each of those stock values it will call the on next method on observer. So this is the observer that will have events from the ticks like in the stock. And then we will subscribe to it and print it out. We can actually do that right here. Okay, so run it. Yeah, it takes stocks despite the poor connection here, cool. And I've got also second method right here which is called fetch async. What it actually does, what I want to show here is that you can wrap a code into an observable, but this time I will call the nq method on the client. That's the asynchronous way to perform the HTTP calls. So the first example we used to execute, right now we use the nq, but the signature of the method is that it accepts the callback. So it basically is the old way of doing stuff via callbacks. So here I am showing how we can wrap the async code that uses callbacks into an observable. So as you may notice, it will return observable stock values. And once the callback is finished, it's just called on next on subscriber. Again, nothing fancy here. But it should be faster because right now it performs lots of calls because then don't wait to each other. And then it grabs the results and sends it to a system out. So let's run the simulation number two. It's much faster. So it can be used for server-side calls, but at the end of the session I will show you another demo how to use it for a UI. Rx is everywhere. It's not a language agnostic model, as I told. Here are examples of the same stuff. So we are creating an observable and interval on it. So it's, as I described, it's a factory method that creates events. And after each second it will create a new number. That's it. And this is just a syntax difference, but it's basically the same across all languages on top of JVM. There is also a slide for non-JVM languages like .NET, C++, Python, and so on. So as you can see, the syntax is different about the... Essentially it's the same. Every time we subscribe to one server, here probably the C++ is the most different code, but it's still the same. It's called interval one second and subscribe. You know? And there are implementations in almost all languages. It can be used even for mobile development, JavaScript, ART, PHP... Find your favorite language and use it. It will be used in Angular 2.x as a UI framework for creating web applications. It should be part of the Xmascript 7. It will be in the JavaScript itself. There is initiative called reactive streams that, it doesn't actually cope with the API, but it deals with the back pressure. It's something, with the back pressure, it's something, for instance, the consumer don't have to be as fast as the producer, then you have to deal with back pressure, so we've got more events on the server side or on the observer, then you are able to consume, you have to some of buffer them, so this framework kind of try to solve it. The WebSocket is pretty good fit for this architecture because for normal requests and response, you basically create an observable with just one event, it doesn't make sense. Okay, then you can create a more complex observable to kind of merge those events into one big observable but still WebSockets, it's kind of natural fit for observables, that's why it has great support in Rx.js, that's the reactive extension library for JavaScript, you just call WebSocket and place the URL and then you have, end up with observable and you can filter, for instance, the topic within what you are interested in. Or you can group by the topic, it actually creates a multiple observers, group by the topic, so it's kind of multiplexing one WebSocket connection for multiple purposes because WebSockets are quite expensive resources, it's good to have open them as less as possible so you can kind of piggyback multiple communications on one channel, right? So let's do a second demo about UI and I will show you that everything is a stream that there are things that you haven't probably thought about it, isn't this way, but there are streams. So here's the code, but I will open it in a browser. So it's a live JavaScript editor, this CSS is not important, it's just how it looks like. HTML, there are just couple of divs with IDs, nothing fancy, let's close it. So what it does, it's just, it's doing drag and drop for an element and it is interesting how it does. So we are using the RxJS library and first time we'll obtain the element and then we are creating a streams of events and this is the concept from RxJS library, it doesn't make sense in different languages that can work with DOM elements, but here if I provide a DOM element and then JavaScript event, I've got a stream of those events from this particular element. So mouse down is a, actually if I press a button on box this will emit an event, similar stuff for mouse up, if I release the mouse button it will create an event. Mouse move, if I move on the whole document it is continually creating events. So how can I program or how can I get the most drag events, again flat map. So we are using mouse down as an input for the mouse drag events. So every time the user press or pushes the button on the box element it does, it maps all those events into mouse moves so instead of this one click I've got multiple mouse moves and I take them until the button is released. That's the interesting, that's the important stuff. So mouse down, I take all the mouse moves until the mouse up is, until the mouse is released. And here this implementation details I'll just want to know how much it was moved. If I press the button and go to other place and release I want to know the delta of X and delta of Y. So at the beginning I will store information where it was clicked and then I will calculate the delta of the mouse drop. Sorry, the mouse up and mouse down event. All right, so we can continue. We can have an event stream called site which is basically we'll take all those mouse drag events. So if I'm dragging the button it will, it's emitting the events and I am mapping them to a function that will return either light or dark depending if it is on the left half of the screen or on the right hand side of the screen. And if I didn't use this operator it will emit a lot of those events. I'm just interesting about the change. So if there is, I don't know, 100 events calling left side I'm just, I just won't one of them. So this operator will kind of compress the stream and I'm removed the duplicities basically. And in other events I will be notified if this drop on which side is this drop. So if I show the console, okay. And then zoom out. I can see that if I cross the half of the screen it is changing the events and if I drop it on some place it will produce a drop event. It's not interesting, sorry, it's not useful but it is interesting to see that it can be used for creating UI. For instance, in the Angular all those currently you have to use the watch event, watch on some variables and it is handled by Colbex but the same thing can be done in observables. Okay, so let's go to the presentation. How to test that? It looks like there is a lot of magic in it and it's not obvious what is happening there. But I like, I like, it is actually a real test coming from the RxJS library again. It is actually ASCII test. There is a, you know, ASCII definition what to test. We created hot observable, hot observable. There will be a couple of events. Subscriber will subscribe at this place and we will use the isPrime function for filtering the events. So only the primes will stay in the stream. So as you may see, expected stream should look like that and then, yeah, just comparing the stuff. If you don't believe me, you can see this real code in their code base. How mature is that? It's pretty old technology actually is not new but there is a high around it right now. It's pretty solid. Netflix is the most propellant of the technology. They use it for the front end as well as the back end. GitHub use that, couple of redhead products. This is our project ocular. Not sure about the Fireflies form but I heard something. Vertex is definitely using that. So just a quick recap. Observer rule are any number of values over any amount of time. That's an important thing to know. Rx can be used for UI as well as in the server side. It's not just an asynchronous call to a server side. Actually the UI part is what makes more sense to me right now because for instance, the Rack.js is the second favorite or perhaps it is the largest hype right now. It uses the Racktiff architecture as well. They call it Flux but it has the same concept as the Racktiff programming. Can be done in another language. It's not language agnostic and called Observer are lazy until you subscribe nothing will happen. If you want to learn more, I suggest there's a course on Coursera given by Eric Mayer, which is kind of one of the leaders of this Racktiff stuff. There's also a thing called Racktiff manifesto which tries to formalize or define what the Racktiff is because every, it's heavily overloaded term and I hear that all the time that we are reactive and what is meant to be reactive. So they try to say what doesn't mean to be reactive and there is also a course, online course in which you can interactively learn Rack. You start with simple examples and you go further and learn more and more. Right, so my message here is if you have an library or project and it does make sense for you to expose the stuff in an observable way, do it right now because it's easier for consumers of the API to use the observable operators and Racktiff streams. Thank you for your attention. Any questions? We've got swag here. Really nothing? Thank you. Just a quick announcement. If you don't have a ticket to the party tonight, there are still a few tickets at the Red Hat booth and if you're a volunteer or speaker and you don't want to come to the party, please go there and let them know so they can give the ticket to somebody else.