 Well, hello and welcome to the next day's Dev Nation live I'm actually coming to you live today from Dublin, Ireland where I was just in Westford Ireland with Clemont But I had to leave him to catch my next flight So he's still out in what Waterford Ireland and you can see what is nice green background there You got the whole Irish thing going properly. Thank you But we have a great session lined up for you guys today We're gonna be deep diving into the reactive landscape if you're familiar reactive programming. That's awesome He's certainly gonna taste and give you some insights on to reactive programming But also the reactive systems architecture that you might be thinking about for the next generation of your applications Clemont is absolutely our expert who goes on talks about these topics on a regular basis He's from the Vertex team specifically But his background is fundamentally focused on how to build reactive systems and I think you're gonna really enjoy today's session So come on, please take it away and let's see what you got Okay. Thank you bear. So let me share my screen Okay Switch to present mode All right, so My name is Clemont Escoffier. I'm working to Vertex Vertex team at Red Hat and today in this session I'm going to explain what does reactive mean because today You have a lot of reactive things and if you google about reactive you will find things about manifesto Reactive system reactive streams, reactive extension, reactive programming, reactive strings And if you dig a little bit deeper you will find that reactive is often associated with many many different concepts such as backpressure or events or elasticity and even spreadsheets But seriously, what does reactive mean? So let's have a look to well to the dictionary And if you go to this page here to the Oxford dictionary and search for reactive you will find this Reactive means that you are showing a response to a stimulus and there is a second definition here Which is that you are acting in response to a situation rather than creating and controlling it so if we take those two definitions and we want to say what is a reactive software we can come up with this a Reactive software is a software reacting to stimuli such as messages request metrics availability or anything Cool, but let's have a look to this Definition here acting in response to situation rather than creating or controlling it That's actually very interesting because is that the end of the flow control Right now when you're creating an application on developing an application you have You do something a's and b's and c and so on so you know exactly when things happen, but When you are doing something reactive things are a bit more tricky because what decide what's next is the next stimuli The next stimulus that you are going to receive. So basically This is the revenge of the across a synchronous system or synchronous API We are in the 80s and so on so we are going to see all this actually is going to change or you build system But just to have some better understanding of what I mean by reactive And what is the reactive spectrum is that we have many reactive things as you have seen so let's try to categorize this So on the left side, we have reactive systems and reactive system has been defined as reactive manifesto We found concept like actor messages resilience elasticity scalability and asynchronous on the right side of this slide we have reactive programming and While reactive systems are on the architectural level or to build distributed system reactive programming is really about as a development model to build event oriented or misadvented and asynchronous program and here's the concepts are totally different who are speaking about streams flow events and of course asynchronous and There is a third reactive things that is very important today. It's right to streams quite streams is a model of back pressure on Non-blocking back pressure and here we have different concepts associated to it like data flow back pressure Non-blocking and asynchronous again What is interesting is that today and since a year or a year and a half these two ones reactive programming and reactive streams are Start to be the very same thing in the sense that the new generation of reactive programming library are Leveraging reactive streams as a basis to handle back pressure. So don't worry. We are going to explain all of this If we are looking in terms of technologies in terms of reactive system today in the Java world We have a car a pure actor model and a vertex For reactive streams, we have a cast dreams Eric's Java version to reactor from pivotal and Vertex and in terms of reactive programming. We have reactor reactive spring reactive extension Eric Java in the Java world and vertex. We can find vertex on these three Different reactive cities and we will see all vertex is a bundle for all of this and all you can leverage from the Ability to use these trees this different concepts. So but Let's go back when I was young and I was doing a very simple application It's almost 20 years ago and my application was very easy I have users that was entering data about products and was looking for Data about this product. So I got a very beautiful form Whereas I was entering data and then I was saving that in a database. Everything was fine. I was very happy. It was working Almost no crash and so on but one day The code base start to be very very Complicating and was growing and we asked me to extract some part of this data inside another services The pricing service that will be under by another team Okay, why not so every time that my application needed a price It will call this service get the price and display it to the user. Okay? Obviously, there is a mirror of this is that my application started to be used by some other services Some others Software from my company or from other company and okay So I had to develop an API in addition to my beautiful form So they can call me and I can give them the products and the price and so on But then users start Well, we're tired to refresh the page to get new data and so on and they wanted to get notifications It's something we have a lot more today While you have a mobile phone and so on you get notification all day long about your mails about messages and so on so They don't want to refresh anymore. They want the data to go to them So we needed to implement such kind of channel and as my application started to be a lot more Complicated in that before we decided to add some logging monitoring auditing and so on so Our complexity and but something we forgot when we did all those transformation is that all this interaction are distributed interaction We are not speaking in such the same Memory, so we are crossing the process space. We are discussing with a database over the network My other pricing services were over the network to my user definitely over the network and so on and There is one thing that we know about distributed system is that distributed system fail They fail for many many reasons Well, there is data loss. There is latency. There is timeouts. There is slowness. For example, my service here may Not be ready to answer my request because well It's other in a pretty bad shape or under reload or under maintenance or something like that My database can become very slow not necessarily because of my database but because of my network I don't control when my user or the other software are going to call my application So I don't know I cannot really predict the load. So it was very very complicated to get all of this Working and the capacity planning was very complicated And what happened was what we all know about distributed system is that my monitoring was telling me that I was dying so the world system was collapsing under the load and so on so The main question is how can we stay responsive in face of failures and under varying workload when we don't control anything and actually That's exactly the purpose of reactive systems. The goal of reactive system is to build Responsive system. So it's an architectural style to build better distributed system based on one primary concept, which is Message driven all the introduction are going to be using an asynchronous message passing mechanism And based on this we get elasticity and resiliency and if we have all three then we get responsiveness This has been defined in the reactive manifestos that you can find here. It's a very nice to read. It's very short So I recommend you to read it. It's really interesting Okay, but oh it works So basically we have all our entities that will interact using Messages and messages are not sent to a specific recipient But they are sent to an address and on the consumer side you subscribe to the to the address And and you get the messages when the system decides that you are the next one So if you have a lot of messages, you just need to scale Or to increase the number of of message you have here Of consumer you have here and it will just dispatch using a wrong or been or something a bit smarter than this If one of my consumer fail well, the message can be processed by another consumer and Something that is very interesting is that the cost the publisher won't wait in an active way for response It will everything is asynchronous so it can continue under other messages and send other messages using a very Few capabilities and memories and CPU everything will find Perfectly well, so when you see this you say, okay, come on everything is simple We'll just do that all over the place And I would just send messages and receive messages and get replies everything Asynchronously and everything is fine But if you try to implement this specific application on such kind of application was not really thinking about it Very quickly you get this you get this Very complicated code with callbacks and callbacks and a lot of threads that are going to try to Conflicts and what happens when threads try to conflict? Well, you get deadlocks or live locks So it's even worse than what we have before so we need a bit a better way to handle this So let's see how we can do that but before doing that Let's explain why it's complicated by introducing you to the asynchronous development model in the ideal world When you have three things to do you will do the first things and the second thing and the third thing and everything We find and you're done here at that point on that But in the real world We have IO and this task a will be blocked because it's calling another services ordering For example my prices or something like that and for that time It's waiting so my thread is blocked and when I get the reply It's kind of I can continue and so on and so on and so on until I'm done and then I can do that to be which Will do us some other IO and so on so in reality You're not completing your work here, but here Way longer and doing that time all this waiting time. You just do nothing. You just wait and we don't like waiting So the actual synchronous execution model is a bit different. It's okay We're not going to wait, but as soon as we have to wait we will try to switch to another task that can make progress and When we're when we can't make progress anymore, then we will switch to another task And maybe when we can't make progress anymore on that task We can switch back to the first one because we finally get an answer So instead of explicitly waiting, we will try to always stay busy So because of this we are back to exactly the same time We haven't get some free time here and to do some something else But to do this we need switching we need an asynchronous programming model We need non-blocking IO and then we need a task-based concurrency instead of thread-based concurrency And When you do that it's in your code things start to be a bit weird Generally, we do that. I have my compute method I pass parameter and at some point I return the result and on the consumer side I just get the result like that, but in an asynchronous world It's not working like that. It's the Hollywood way We pass the parameter and a callback and once the method has computed the result It will call you with a result So on the consumer side if I'm using Java 8 lambda expression I will use compute I pass my parameter and then this will be my callback here And it will be called when my result is there Everything fine. Why not? Let's have a look So if I'm using vertex to create an HTTP server as an example Every time that I have an HTTP request, I will call this handler here Okay, fine and every time I have a new request. I'm going to call this again and again and again To start my HTTP server, I will use this method here and this callback here will be called once HTTP server as Actually is started or fail to start because obviously there is many reasons why it can fail. So Looks nice. The code is okay. Okay, but let's have a look to another example. I Want to retrieve data from a database. So I need a connection This is going to be asynchronous because getting a connection is asynchronous Then I need to do a sequel query. For example, this is also a synchronous So I'm need to do another callback and obviously when I'm finishing processing the data I need to close my connection. This is also a synchronous and a start callbacks So we need to phase it. This is not cause that I want to write and read Everything every day because even if this is relatively easy, it doesn't scale and So all the vertex ecosystem is developed like this and believe me, it's not easy every day Well, we do that because we don't want to depend on anything and we want to have something Homogeneous but yeah, it's kind of sometimes it's hard to read. So what what can we do? How can we fix that and it's where we have reactive programming and reactive programming actually Has been designed a long time ago And the idea is how do I write? code to under asynchronous events and streams But to explain it the best example is spreadsheets So I do a lot of meeting conferences and so on and the most painful experience is my expense reports But fortunately I can use Excel to at least prepare my expense report and this is reactive programming So I have for example lunch, coffee, drinks, and this is the total But if you are an Excel user, you know that this is actually a Formula that is expressed in terms of the other cells Right. So there is an observation. So as soon as one of these cells change These cells observe the change and we compute the new value And over time we have something very interesting happening because at the at the beginning I have only my lunch 15 euros. Okay, so this become 15 and then answer 25 coffee because of that like I need a lot of coffee So this is 40 euros and then a lot of drinks and then I'm at 85 euros so over time a cell is not one value but a sequence of value and it's a stream of value So the stream of value depending on the technology as a reactive programming technology are going to use I have different name. It can be name observable flowable flow or publishers depending Depending on what you are using, but if we are using We are going to use a rich Java 2 and we are going to use the flowable concept But to give another example because that's what reactive programming is is about streams let's have this red stream here with the sequence of integer and Then I want to express to have another stream that take all those data and just do plus one That's my green stream So every time that I have a new data in my red stream I have this a new value in my green stream and let's do something a bit more Complicated in my blue stream, which will be which subscribe to my green stream Get the two last value and do the sum of it for any reason whatever if I want to express this Using Java for example Eric Java 2 I will do like that First I have a stream named flowable of integer, which would be all the integer from one to ten and then I have my green stream Which will be subscribe or observe the red stream of the one and map all the value to The value plus one Okay, why not so this will be my green stream and will be exactly this so for one It will give you to give three and so on and then I have my blue stream Well, I'm subscribing and observing the green stream get the last two value and every time that I have two value Just compute the sum of it. So it's actually interesting what happening here that here we have flat map because map associate to one value another value while flat map associate one value to a Sequence or a stream of value and just flatten this set of value So he has the sum of two integer is just an integer about the API is like that and don't forget this This is probably the most disturbing things when you start using a reactive programming if you don't subscribe Nothing is going to happen. It's very lazy So you need to say oh by the way all this transformation as a purpose and the purpose is that I need the result So don't forget to subscribe to your sequence of stream So here I'm subscribing to the green as a green one that will Then enable the blue ones and enable the green one and then also the red one and everything will start working Okay, so but what's the relation between this and the asynchronous senior was mentioning Actually because the stream is inherently an asynchronous structure For example as flowy ball observable our stream of data. So every time I have an HTTP request I will be involved and I can do transformation on it There is another type of stream which can have only one value Like singles or zero one value like maybe and this will be used as a second operation. I need to call my price of service I don't know when I will get the reply. So but I know that we have either Response or an error and this is what the single type is made for We also have completables which are for asynchronous operations that have no returns I'd say, okay, tell me if you have completed or not like writing somewhere in a file and stuff like that and it's interesting because You can use those reactive types of flowy ball observable single maybe incompatible to really Write and develop all your asynchronous System but we will see an example of that in in a minute So typically If I want to rewrite my gdbc or my database interaction First I need to get a connection. So I will use this method here that gave me a single So a stream of one with the connection Once I have my connection I say, okay, once I have my connection Execute this query here select star from products or get all the products Get all the stream of row and transmit to a flowy ball so to a stream and when I'm done iterating over this stream Just close the connection fine So and for every row I got I will create a new product using some kind of or I'm mapping or whatever and I just subscribe to that That's exactly the same code I was showing with my sweet nested callback and now we have a way more Declarative way to express exactly the same thing and what I mean is actually this we have the singles and I Do my query I get my query here. I got my sequence of row and I map it to product and then I've subscribed here Is that all working? Wow? Maybe something is not that simple actually and we are going to see that now What up on if at some point here? Something very slow is done like instead of just displaying the product on my console I'm actually doing very heavy computation on it if I'm in a blocking environment What is going on is that it's going to block my thread till the beginning But maybe you can't do that because maybe The threads that is producing the data is not the threads that you are using here So what's going on in that case? What's good? What can happen one when you're processing can't keep up? What happens in every case is we are going to fill a buffer and what happened when the buffer is full It's going to crash out of memory or in Eric's we have exception for that, but yeah, well Buffer are limited. So you need a way to handle this if you Can accept to lose that are well if I had that allows is acceptable Just say it and say, okay, my buffer is only three and As soon as it's full all the new data I would just read your exam to slash the stuff now adjusting your them. So obviously you can tune it to have the last that is drop or the first that is drop and so on but That only works when data loss is acceptable, but what about when you can't have data loss so Then we need a protocol Between the consumer and the producer So between the two streams to say well, I will tell you when I'm ready to receive and don't give me any data until I'm ready This is actually what is done by Ricky streams, which is one way to handle back pressure in a non-blocking way and all it works is that My consumer stream here is actually going to ask this stream to say, okay, I'm ready. Give me X element The element are will be produced asynchronously when they will be ready and When they are processed this stream can say, okay, I know I'm I'm back on track Just give me another set of X element and so on. So it's interesting because all the stream Semantic is about pushing data to the next one entry here We are reversing it to say no no the consumer need to say when you are ready and so But it's mandatory if you don't want to crash and to collapse under load or on the way too many data Actually for you if you're using Alex a job at you or rector You don't need to care about it because that what publisher or flowy ball and implementing Beyond the scene Floy ball and publisher. So actually publishers are active stream type and flowy ball is an implementation I'll just Well, I'll just stream Implementing this semantic so you don't have to do it. It will just work out of the box Well, if you're using observable then you're on your own and you will have to handle the bike pressure as you can using your own protocol Okay, but let's know I'll have a look to to vertex a little bit more and explain you why it's interesting for you If you are looking for reactive and it will give you a better understanding on all all those concepts work all together So reactive a system or active streams and reactive Programming so what is vertex eclipse vertex is a toolkit to be distributed and reactive systems It's just a library with a large ecosystem and all this ecosystem is reactive and it has been designed to be reactive in mind It's not something we have added after as an afterthought The first line of vertex was reactive. So we learn a lot by doing that. What are the issues? How to handle blocking code non-blocking code and so on It's proposed not one but several asynchronous non-deblocking non-development models If you want you can use callbacks you can use every Java one You can use every Java two, but it's also polyglot and you can use a scalar Kotlin Groovy Ruby and Ceylon and we say polyglot It really means that the world vertex API is generated for all those language using the constructs from this language for a natural integration We have a simplified concurrency model based on an event loop So you don't care about concurrency You don't care about thread is going to use a minimum number of threads and you will always your code is going to be single threaded But as you are not looking it's not big deal. It's going to work And it has a cluster event bus and we are going to see that that will enable the reactive system Vertex is used today from micro services web app IOT API gateway I volume event processing and many many other application and if you're interested go to HTTP No vertex.io and then scroll to who's using you will see many Companies are using vertex in production today for very different use case So why it's interesting in terms of reactive it's because Vertex was made to build reactive systems. We have reactive programming and we implement reactive streams So you have all three in one bundle Everything come from free. So let's have a look at what it looks like in a real application So and remember this application here. Let's implement it in vertex very quickly. So Let's focus on this iteration first Oh my user get my product although how do they insert my product? So let's imagine that my database as this API here insert that return a single Eric's Java 2 singer When the insertion has been done, so I got a request I do that on my database Okay, that's what I have here and when it's done when the insertion has been done I just reply the two hundred ones or created a A stupid response and if something bad happened because the database is not there or time out or I just fight a request To list so when my user want to list the product, it will just say database that retrieves. This is return a stream of products Okay, so with all the products and And then I subscribe and something interesting here is that I don't want to Overload my memory with all the wall set of products that that would be very inefficient in term of memory So I'm using something that exists since HTTP 1.1 is chunk responses As soon as I have a product, I'm going to write it down inside my HTTP Response so meaning that my memory will never blow up. Oh by the way, I forgot to say if you're interested by this card It's there. So you use this get a repository So Yeah, as I said as soon as I have a product that come back from my database, I just write it to my HTTP response and And it's my client side that will have to handle and display it correctly Okay, but what about calling my price here? So let's have this first version where this is a web or a Rest service or an HTTP endpoint and I'm going to use the the vertex web client for this and When I retrieve my product so this interaction here for all the product I'm going to call my processor using my web client. So it's just a vertex HTTP client. So I get the price I Extract the price from it and I produce Product of that and the rest of the code doesn't change Pretty cool because for each product. I'm going to use a web client fine All is non-blocking and asynchronous and everything is fine And I've as soon as I was the first response or the sequence of response I'm writing inside my HTTP Response the product and so on using my chunk response in The issue here is that I depend on one price or service and this is not really what we had in mind with Reactive system where we need to send Messages to an address and not to a specific recipient vertex come with a cluster of event bus and instead of using the Web client and so pointing to a neural we can just say hey Just send it to this address here and waiver is listening to this address is going to reply to that one To my message in an asynchronous way and the same it's going to be the same Type of interaction except that no we don't hot-cold the URL We just use an address and we can have one instance or x instances of my pressing service It's fine. Everything is fine and the rest of the code stay unchanged Let's continue in building this application here and say yeah, we want to inform our users as soon as someone's has added a product oh But he needs a price too. So what can we do here? You say, okay, I'm inserting I'm inserting my my product name. I'm retrieving the price of this one. Okay, and now I have my product with a price Writing the 200 a stupid 201 HTTP response to say to my To the one writing it that okay, it's done But I can do an additional things because this is not the last The last instruction I can also say oh and I will announce this new product to everyone listening on this products address and The browser from all the user can also be part of the event bus using what we call event bus bridges And so we see those Messages meaning that using a suggest channel Receive all the new products was not having to reload it. So oh, oh this it works So suggest is a JavaScript library that will Negotiate the channel between your browser and the application using web sockets and then degrading to SSC and Long pulling and so on depending on the capability of your browser Okay, so and also the last thing we were wanting to do is To do to call the audit system here But what I can do is get the price and call the audit at the same time I don't care to do that in a sequence. I can do it at the same time And this is what is done by these three lines here. I guess a price from product. That's a single It doesn't do anything. Nobody's subscribed to it. That's my action to send the action to to to the audit system Okay, again, it does nothing then I zip both stream meaning that when both are done, I will continue That's a stream and then I subscribe on it and everything will will be done will be executed and Both interaction happen in a second this way in parallel Well concurrently so it's pretty cool because you don't have to think about it about thread and so on everything is Is there and works perfectly well? So let's run this so again, I'll show you the the GitHub repository is exactly this code and I'm going to run this as a price or service is normally started fully. Yeah We'll see that pretty quickly. So again, that this is toolkit So I just wrote a main deploy my vertical so a vertex component right-click run don't need any Plug-in or anything? Let's go here That's this one here and yeah And it is so we got my products that are well in my database and Let's try to add one but before adding one. I will copy the same URL Open Safari here And we should have the same set of products. Okay. No, I just want to move you. Okay, and now I'm going to My terminal can't get access my terminal from here or I just stop the screen sharing It's not very cool yet. Sorry about that. I will open a new terminal here Where is my Safari so that you can see everything at the same time. Okay? Okay, and if I do If I want to add water for example Boom, you see that what happened is that this one got the 201 response as we said, okay But also publish the application publish on the event bus The new product and send it to both Here and here at the same time because both browser are connected using so GS to my event bus It's a very easy to do any kind of interactive application like that because you don't need to care about Who is sending the event and so on here? It's I have one application But if you are in a microservices system any of your microservice can send an event on the event bus and Also a connected browser can get it Let's go back to my slides here all right and Let's conclude no because I'm oh, yeah, I'm a bit out of time so what does reactive mean reactive is just a new way to build a synchronous system, but All all the world is a synchronous everything you do is actually a synchronous I'm speaking you and you're receiving all my word Asynchronously when you are typing on your keyboard is asynchronous your operating system always dealing with Interruption and so on is asynchronous too. So it's just another and very powerful way to build such kind of asynchronous systems So reactive as many many facets and the main three ones are reactive systems on one side, which is Well, let's say distributed system done right to build responsive distributed system based on a synchronous message basing and elasticity and resilience But implementing such kind of system can be hard So we have reactive programming that will help you to build such kind of system at the core lever How do you develop it on a daily basis? how do you manage all this stuff and reactive streams is actually very important when you have streams because When you have something in your stream process that can't keep up You need to slow down the world chain and not just an previous one, but the whole chain and it's all Our reactive streams can help you to build that but again if you are using a modern reactive programming library No problem because it's hidden for you You don't have to do anything. You are just using flow flowable depending if you use vector or Java 2 and you're done That's all I have if you're interested to go a bit deeper about vertex and about Reactive and microservices. We published this book a couple of months ago You can download it for free from this link here or from this link here if you want to scan it Well, I'm joy and if you have any question, I don't know it was come on That's awesome We do have a couple questions for you though at least one that I think is super important because it comes up all the time And that's related to back pressure So is for instance in your last chart there is back pressure specific to reactive streams or can I implement back pressure and see it in other ways So that's actually a very interesting question And if we can't discuss today like like that if I can present it's because there is back pressure in the wire TCP which is the protocol implementing internet today has actually back pressure and So by pressure is not a new thing and that is since the beginning of computer systems and network systems Back pressure is a software way Reactive streams is a software approach to deal with non-blocking back pressure But there is many many other model typically in vertex We have to we have reactive streams and we have another model based on a pause and resume So when you're full you pause the stream Continue processing and when you're back on track, you just resume the stream So you can you can implement back pressure differently Very different way, but and I think that's probably a key point You have to have back pressure support both at the consumer level and and producer level if the producer Doesn't care about your back pressure. It will just ignore the message. Yeah, please slow down. Please stop and actually today In there is one way the most common way to deal with with by pressure is to use Kafka Kafka as an Unlimited buffer or unbounded buffer where you've write her with right inside and you consume her with consume the data at their own pace So it's kind of a back pressure protocol just using an Unbounded buffer in the middle name Kafka Yeah, I love that point and your demo by the way is totally awesome I love how you can see where an event happening in one part of the system is immediately visible throughout the rest of the system It feels like the nervous system of the body human body kind of thing, right? The event may immediately bribles out. Your body is reacting to stimuli my system react to stimuli of the same thing We're just mimicking the biological world Okay, well, that's all the time we have for today. We always like to keep these things short I thank you guys all so much for showing up today as you saw in the chat if you're participating in the chat We have people from all over the globe and that's absolutely amazing That's certainly why we do this to make sure that this kind of content this kind of valuable Information coming from come on today can get out to the world at large And so thank you guys so much Please feel free to download those books check out the recording here on YouTube If you have any questions or concerns hit come on or myself up on Twitter or feel free to email us back on the same email You got from us when it Alerted you to this session. So thank you for participating in Dev Nation live today come on. Thank you. Yeah. Thank you very much