 File name in newtwsied? I can have a demo so there moment at 15 minutes Welcome to the talk, and I am talking about transactions for theounge Today I am going to talk about micro services translation is appropriate for the running of micro services in dressing up environments Yn dwl 3 maen nhw ddim ychydig yn c pleadidio o gweithio tuning wedi gwneud ynnydd ser achi sydd o bod pan gofaws répers. Os yna, wrth gwrs ddigon nhw'n emrwy'n Sound Stewart yn un arworldgell iawn. ac mae'n gweithio'r ffordd yma yng Ngheilio APEI'r Ffwyrdd i'n gwybod i'r parw sydd i'r parw sydd yn ei wneud i'r newid yn ymwneud o'r cyhoedd yma o'r ffordd ar hirio ar LRA. Mae'n gweithio'r ffordd o'r llwyddoedd hynny. Mae'n sgrifau. Mae'n gweithio ar gyfer, ac rwy'n gweithio'n gweithio. Mae'n gweithio. Mae'n gweithio'n gweithio ar gyfer. A'r hyffordd yma yw'r hyffordd? Mae'n gweithio ar gyfer. ..eith yw'r dda'r cyffredin? No, mae hwn yn dweud, hefyd. OK. Yr eich gwasanaeth ar y cyffredin. OK, yn ystod? Efallai yna'n mynd i? Efallai, mae'n gweithio'n cyffredin. Ie, mae'n gweithio'n cyffredin. Fel gyfer. Efallai, efallai, efallai yna'n mynd i? Efallai. OK, dwi'n mynd i'n meddwl ymlaen, mae'r cyffredin a'r gyfer y system yn cael eu cyfle. Mae'r cyffredin a'r methu ynghyd, mae'n ddweud yn ymdegu yn buddwl. Felly, mae'n dod o'r hyn, mae'n defnyddio'r ddweud yn y ddweud yn hyn ymddeg, mae'n ffordd yn rhywbeth, y dyfodol, y dyfodol, bod yn fan ymwybod diddod. I yw'r gweithio rwy'n amser yn maes ei ddweud, ond yn y digitalr hon yn ymgyrch. Fy hoffwn ymddeg ar gyd, fyddwn yn fyddi, ymdeg yn ddweud. Fyddo'r gwylltion. I wnaeth ymdeg ar gyd, mae'n gynyddu mae'n gallu gweithio cael ei chael yn ymweld, i wneud ychydig yn gyllidol yr horddiad. Yn ystod, mae'n cael ei gweithio, rydyn ni'n gweld, ymweld, fe, mae'n ddigon y system cyllidol, a'n gweithio'r rhaid o'r gwybod, a'r gweithio'r ddweud ac yn dweud, a'n gweithio'r gweithio'r gwybod, a'n gweithio'r cyllidol yn ei wneud. Ymweld yma am ysbytiau yn ei wneud. Ac yn ymweld, mae'n gweithio'r cyllidol. Yn ystafell sydd y dystiolaeth, mae'n gweithio'r problemol yw'r ysgol yn ymgyrch yn ymdwyaf. Mae'r micro-surfadau sy'n dweud y dystiolaeth. Mae'r cydnoddau yma, yn ymddangos chi'n gwneud, mae'r cydnoddau yn cael ei fod yn siarad, yn cydnoddau'r cydnoddau. y phunwyr yn oed y fathodol rym ni'n gweithio mecanallwydau o'r ffaenonon eifryd o gael mwerth y llwyd gan y tîm yn rhyw fan. Mae'r cyflawnsion gwahanol, mae'n cyffredinolaeth i'w dweud o gael y ffaenonon i'r dda. Ond ydych chi'n gallu nhw'n cael ei wath dependr, yn ei wathwn i'n cynhyrch a chanol, y syl passwords, mae'n anodd wneud o ceisio i un pwysigol ar gyfer y chaillol sy'n mynd i'r gorffwnol o'r maen nhw'n wneud. You've got tim Aus. You can do load balancing. If you detected that one of your loads is running more slowly in scale up and down, depending on if you think that things are becoming fullacting or you've got latency in your system. So we think that transactions is really part of this ethos and that should be able to be part of your tool case. It's not the solution, but it should be one of the solutions. You can apply it to building your systems. Ac y cwestiynau eěn meddwl, y wybodau y diwrnod i ddim yn gynnwys, ac mae'n meddwl ychydig yw'n bwysig hyn? Mae'n meddwl. Felly, mae'n meddwl mewn gwahanol i gyd,. Cefyddiadu gynnwys yma, o'r ganddo ar feddwl y shawdd y dyma, ai'n i'n mynd i'r gwahanol i gyd. A'n meddwl, y dyma eich ganddo â'r ganddo y dyma, nad ynddech chi fod os y gallai dechzefnu.ressing this guarantees. So Ifeles weld it provides like an all-or nohting compliqué so you like if you move the system and so thyn Gridio gwaslawr at the end of the transaction you want to be able to revert things. So there's only 1 model of transactions there are many different ones so you've probably heard of the GTA model which is like built based on an old ex open of A'r model JTA yma, mae'r sleif yn cael ei amser yn ar gyflogodd. Mae'r model yn cael ei amser, mae'r model yn cael Saga'u, mae'r bwrdd amser cyntaf amser. Mae'r busnes ac yn ei gweithio ar y dyfodol, ac mae'r rhai gwneud o'r bwysig o'r protocol. A bod eich bod yn dweud, ydych chi'n oes y transasktion model? Le'r modd Llywodraeth wedi bod oedd y modd fynlluniaeth neu'r modd Llywodraeth, fel'r modd Llywodraeth yn ymddorol gan мол Apparaturi. Oherwydd mae'r llyfr yn ymddors fawr ac'r modd Llywodraeth wedi'i amser gan y gynhyrchu Llywodraeth. Byddhaf i'r newidau ddechrau yr un iddyntau ar y modd Llywodraeth, rydyn ni'n enw i'r ffynlluniaeth gwahanol o'r modd Llywodraeth a'r rhaid i'r moddd Llywodraeth. I'r modd Llywodraeth yn ymddors fawr. Bydd y cyfle其实a daeth eich sefydliadau a fyddwch chi'n gofio'r cyffredin iawn. Mae hyn yn mynd i bod yn bobl cynnig iawn maen nhw'n aethom. Felly, yna, y gallwn i'w gweithio'r cyffredin iawn, neu'r cyffredin iawn, y gallwn i'w cwchio'r cyffredin iawn. Felly, dyna'n gweld hynny sy'n gwelch chi'n hoffio'r cyffredin iawn, felly dyna'n gweithio'm yn cael ei bwlad o'r model ei fawr.. Felly, mae'r model eich bod ynwn i'r ddweud allan o ddweud o'r ffordd mewn ddweud o'r ffordd i'r ddweud o'r ffordd o'r ddweud o'r ffordd, ynnyddio'r ddweud o'r model jta. Felly, i'r ddweud hynny, roedd hynny'n gwneud o'r ddweud o'r ddweud. Felly, atamysgwyd o'r ddweud o'r rhan o'r oedd o'r prifsfyddiad, mae'n rhan o'r prifsfyddiad o'r rhan o'r sengl uned yw rhaid. Nodd hynny, when you finished on a set of operations, you want all the operations to complete or none of them to complete. So that would be full atomicity. Second property is consistency. So that's an application developer's point of view. So he's building his application and his business application provides certain guarantees and certain invariance. Before you start running the transaction there's an invariance that you can rely on and at the end of the transaction is moved to the states moves to another invariant of the system. And that's saying that the system is consistent from the application's point of view. So in between the start and end of that transaction then you can have inconsistent states. So that's why we have to introduce a third protocol, isolation. So you want to be able to isolate that inconsistency from other transactions running the system. So two transactions cannot see what each of them are doing in between the transaction. i gychwyn eu cyfrifiadau sy'n gallu ei chygingrifio i'r cyfrifiadau gyda gwpaeth. I fawr, yn yataeth, lle wedi'i hen i gwybod y cyfrifiadau cyfrifiadau yna yn ymwneud eich cyfrifiadau gyfrifiadau y gydag ar eich cyfrifiadau. Mae yna dwych i'r cyfrifiadau, 어� i gydig i'r cyfrifiadau, ond mae'r ei wneud am y cyfrifiadau i'r cyfrifiadau, a bwg yn gweithio ac dyna fydd y pwynt sy'n oes chi'n gweld â'r cychwyn. Felly mae hynny sy'n mawr ysgwrs cychwyn swyddfaen, yn cymdeithas y fawr i bobl ei fawr yn dweud. Felly faint o'r hyn yn oed, mae fan hynny'n meddwl eich bod adeiladol yn chi gweithio a w abandon i bwg yma? Maebell a'r pwynt hanes o golygu ar gyfer rhai ffARS, mae'r cychwyn cychwyn cychwyn cofwilion wrth gwrdd y log yn cyd-deithas hynny rhywfydd. mae'r dŵr ymlaen i fyng o'r rhaid oherwydd mae'n bai yn ychydig o'r pethau sydd yna'n gyfrins. Felly mae'r gwrthodd neu'r ddweud o'r ddafodd, ddim ddweud o'r ddweud o'r ddweud o'r ddweud a phobl iawn i'r ddweud, ac yn ddweud o'r ddweud o'r ddweud o'r eu ddweud o'r ddweud. Felly mae'r ddweud o'r ddweud o'r ddweud o'r ddweud o'r ddweud o'r ddweud. Felly, mae'n ffwrdd dros agnod o dollars i chi gynnwys a'w'r pethau sydd yn fawr ac i chi gynnwys pan ddoch. Felly, mae'n gyrfaen â'r f hyn mae'n cyflymau fawr i gyd yn dwylo'n amser dyma sydd yn rwyf yn gweld bwrdd syd. Felly dwi'n gyrfaen â'r ffordd teiemol iawn. Felly, eich ffordd ffyrdd ymweld yn dweud o'r peth sydd yn dweud â gwmwys gyda chi'n y gwirchai o'r mae'r wlog gofyn. felly mae'r ysgolwyr mae'r ysgolwyr wedi'i gweithio'r ysgolwyr, dwi'n rhoi'r ffordd o'r ffordd o'r ffordd i'r Lymbr. A wnaeth i'r ffordd o'r cyfnodd, o'r gwneud o'r ffordd o'r acedau acedau, mae'n rhaid i'r ffordd i'r gweithio'r data ac yn rhai o'r log. Felly mae'n cael ei fod yn ei wneud. Felly y ffordd o'r gweithio'r ffordd i'r wneud o'r log o'r ddweud o'r ddweud o'r gweithio'r data. Felly, mae'r ffordd i'r gweithio'r ddweud o'r ffordd i gweithio i ddeudio hynny,worda, mae'n d possessedio o explanation, i'n gweld sesi's Pressing to have interior of the ar gyfer y partid yn ymdod. So that's... So what about using transactions for microservices. So microservice interactions typically what you do is you start with a monolith. You've got a monolith that's running inside an application server and you want to break that monolith up into different services a goes on run those in separate machines in the network in the cloud. For a start, you've got many different parties involved, so that increases the complexity. But once the system's actually broken up, when you're talking to Microsoft's environment, them will be a strong propensity toつ and do use plug in other people's APIs, like them so you might like, you have to only concentrate on your core strengths, ond wrth gwrs y tro rydych chi ar gyfer y sylwb i gyd沒關係 ac yn gyf feelingwyr yn fwy o'i oed i'ch gynghyd i'r lleidio a'r lleidio'r lleidio. Mae hyn yn gobeithio, a'r hynny'n gobeithio, oherwydd roedd gan ffacolig sydd y migrosirfysg ac yn cyfnodd y migrosirfysg sy'n byw bepysig, dyna i siarad yn cael ei gydag i'r gwell llunodol, ond holl o'r tanig iddyn nhw. O cael ei wneud yma yw i ddweud o ddych chi i'r gwas iawn i'r ond, you need something to be able to co-ordinate all these different activities. Otherwise, the system's going to be like that because you're going to have to have a lot of requirements and a lot of responsibility on other services because of his trust balance that you don't really have any influence over. So you need some kind of thing to co-ordinate all of that. So the question really boils down to can we use full asset transactions to achieve that correctness. Felly, mae'n debyg arall oherwydd mae'n byw gyda'r syniadau dystrybu'r system yw'r 60s a 70s, ond mae efallai i'r wneud o'r modd yn cymrydau yn cael peth. The problem with full acid is clearly that two phase commit protocol is a blocking protocol. Also to achieve the isolation typically you have to provide locks on data although typically you like your locks on data zone if you use optimistic locking during the transaction. Then at the final two phase commit phase part of the protocol that's when you do the locking. pan oedd eich cymdeithas yn y pwg yn ei ddweud. Yn dweud o ddweud, yw'r cyffredinol sydd wedi ariedd o ddweud y cyfan llwyddi, ond mae'r cyfeirio'r cyfeirio'r cyfan llwyddi yn y pethau yw'r cyffredinol. Mae'r cyfrannu cyfnodol yn ariadur. Mae'r cyfrannu cynnwys i ddweud sydd wedi cael y set ymgyrch, mae'r cyfrannu cyfrannu, If you've read one bit of data during this transaction and you read it again, it's guaranteed to be the same value. You've got to be committed and read uncommitted data etc. So the highest level is serialisable. So that's serialisable, that would make effective meaning that if you run two transactions it would effectively look as though they've been run one after the other consecutively. And that's the one that impacts availability the greatest. There's also a piece of light, a theorem from an academic literature called the CAP theorem. So the CAP theorem concerns about consistency availability in the presence of partitions. So if two services in the environment if they lose connectivity because of a broken network or because the machine has gone down or because the service itself is running solely, then that's what's called those two services become partitioned. So if two services become partitioned then the CAP theorem states that you can't guarantee full consistency and full availability. So in this kind of system you have to be able to relax one of those if you want to be able to make progress. And in a microservice environment availability is key because that's why you're going for the microservice to get the scale and to get the responsiveness of your systems. So really it's like down to like looking at how you can like what you can do with respect to consistency. So full assid is clearly it's not an optimal solution in these kind of loosely coupled systems where you've got long duration activities. So if you can't use full assid what the option is is to start looking at those set of four properties that characterize all transaction models and not thinking about which ones are those you can start relaxing or even do without in some cases. So as I go through the four assid properties to make it concrete I'll refer to a simple case of where you might be trying to book a seat on a plane and some travel insurance and you want to do those in the side of transaction. A conventional two phases in asset full asset transaction you would have to book those two things as a one atomic operation. So either the both booked or neither would be booked. So if you want to relax acid a bit atomicity after that all or nothing guarantees then you might want to need transaction. You might want to like cancel some work while allowing other work to continue. So in the example we've got the travel insurance example you might go away you might book the flight and then you don't want to like unbook the flight if you can't get your travel insurance because you might better get your travel insurance some later time down the line. So that's a case where you want to break out atomicity. So the more conventional approach to handling that kind of break of atomicity is with nested transactions. So you don't get nested transaction with JTA but you do with a lot of other models of say like the object transaction service from CORE for example they had nested transactions. So that's where you can do some of the work inside the nested transaction and then if you decide you don't want to do that nested work you can cancel it and it won't affect the top level transaction which will continue running. So if you book your flight and the top level transaction that's still there and then you just like cancel the work you've done with the insurance. The second possible property to relax is about consistency. So you might have heard in terms of when people talk about transactions for the cloud and for microservices environments they talk about the idea of eventual consistency. So as long as the system eventually comes to some kind of agreed consistent state then that's often satisfactory for a lot of applications. But eventual consistency is a very weak guarantee. So when the system and the data will converge on a consistent value that's indeterminate. It can happen at all times in the future. Also once it's converged it can quickly start diverging again. But that can be mitigated with application knowledge so typically you would combine it with timeouts. So if you're waiting for some state that you want to be consistent and it hasn't happened within a certain period of time then you can time out that work and go and try a different approach. Try a different strategy for 15 minutes or abort it completely. So the next one is isolation. So when you start. So as we said earlier I think into isolation you have to be able to relax isolation because isolation requires a lot of light work on behalf of the application developer to isolate work from the other applications. So you might want to commit work early. I'll skip through that one. And also the durability aspect. So typically you want your work to be good even to be durable and to pierce after you've completed your transaction. But STM software transactional memory that is a good example of where you would like to relax durability. So in a software transactional memory system that has like the acid properties that has like atomicity consistency in isolation. But it doesn't have the durability and typically you'd use that for like high scale concurrent systems. Object based systems. So you've got your objects and you're making changes to your objects with many concurrent threads operating on these objects updating state. And then when you come to commit at the end of your transaction you find that there's been a problem. Then you can just abort it and start again. So in that case you wouldn't necessarily want to do durability. So how do we go up route choosing which set of properties to use? So we approach the micro profile community. Disgusting with them came with some of the odd use cases. So we knew that scalability was the main thing to look at. So full acid wasn't proposed. We needed to remove locking so we need to like something that's where different business activities can see what each other are doing. So you need to drop isolation. Basically you want to make sure you get as much work done and much forward progress as possible before backing out. We knew we wanted nested transactions. We knew we wanted to be a time bound. Time bound the operations. Time bound the period of which the undue compensation activities can be guaranteed to run. And also composition of these transactions called long run actions. And also to be able to compose them and run the compensations within long run actions as well. So the result was a draft specification. So we submit that to micro profile community. We have a weekly meeting where we go through all the different stakeholders, go through the different issues and different problems with the spec. So it's still open. It's still available for people to make contributions to it. Micro profile is the CDI first environment. So we define a set of CDI annotations by which you can start these long run actions and by which you can register callbacks for the compensation completion activities. There's also a pure Java API for people that don't use CDI annotations. And also we've got a group that have their own implementation of a saga-like model that uses GRPC and Google prototype buffers. So we've separated out the transport aspects from the CDI annotations that define the model. So this is a sequence diagram of our Narayan implementation of it. So the actual specification is defined in terms of like a set of CDI annotations, as I said, and the map of the transport. So this is how we implemented it. So I'm going to skip over that one and go straight to the demos. So in the demo we used various pieces of technology. So we've used the OpenShift platform, OpenShift, to build on top of Kubernetes. It fried some extra management functions on top of Kubernetes and also the build and CDI deployment pipeline. We've also used the Narayana. Narayana is the Transaction Manager Toolkit, and we use that to build the prototype implementation of this specification. And also the Wildfly Swarm, which is now called ThornTail, but they haven't got a logo for it yet. So ThornTail is like a way to get a cut-down version of Wildfly, the Wildfly application server. So it's cut-down, it has just what you need inside. It affects a bit of a piece of the fat jar. That you can run in a JVM. So this is what the demo is. So the demo is booking a hotel and booking a flight. So the initial state is to book a hotel. To book the hotel, we run that in a top-level route lowering the action. And then the flight bookings, we want to have two strategies you want to book. So we've come to book the flight and we can't find an economy one. So okay, we'll have a first-class flight, but we might want to later go back and cancel that and go back to an economy class. So we've run that in a nested, in a nested lowering action. So the idea there is that you can go and cancel the first-class flight and then have another go at booking an economy flight. So you're not going to lose your hotel booking. And then after we've done that, we're going to cancel one of the flights and then close the LRA. And the thing that's managing these LRAs will then make sure it calls into all the different services that are registered with the LRA. So for example, if you cancel the whole operation, then the transaction manager will ensure that it goes back and calls all the different callbacks to tell them to compensate for the work they did. So the hotel booking could be to refund the customer. So there's a deep dive there. If you do want to add these slides online, if you want to have a look at what we've done. So I'll just have a look at running the demo. How long we've got now? Nine minutes. Nine minutes. Okay, so quickly. So yeah, what I've got there, this is the set of annotations. So for example, to start an LRA, you would annotate one of your methods with an at LRA annotation. So there you've got things like required, requires new. If you're familiar with JTA, it's a model on JTA. So with those annotations, so for example, if you put a mandatory LRA annotation on a method and there's no LRA context available when that method is called, then it'll throw an exception, for example. Okay, so let's have a look. So we've got... Yeah, I'm running a balance show on time. So I think it's probably best if I just run the demo. Right, but first I'll say that so the three services running is the trip service. So the trip service is one that does the booking, that coordinates the booking of the hotel and also the flight services. So then there's one for the hotel service and there's another one for the flight service as well. So if we start, the service is running. So this one is the hotel service. This one's the flight service running. So this is what I mean by the flat jar. So a jar of a minus jar and then there's the flat jar is like a thorn tail. It's like a cut-down version of a wildfire flight. And that's got the bundle inside it. It's the various services that we're running. We also need a coordinator. So when these microservices register compensation activities with the LRA, that they might want to be compensated for if the transaction is subsequently rolled back, closed, cancelled, sorry, it needs to register this information with the coordinator. So we've got two coordinators running in the same environment. That's the bottom right one, that's the one for the sub-transactions and then we've got another coordinator. So the coordinators can be federated in the system. They don't all have to be running in the same JVM because they can talk to each other. And then this is the main coordinator. So when you do a trip booking, the trip booking is going to go to the main coordinator and register with that one and then the flight service that does it in a nested LRA. That one will go and register with the subordinates, with the subordinate coordinator. OK, so that's the services all running. So what we've got now is... So this shell script, so this shell script is going to... is going to go and make a call to the trip service microservice and it's going to ask it to go and book a hotel and book the flight. So if we run that. OK, so that's gone and done the booking and that URL I've printed out there, that's the URL that you have to give to tell the trip microservices to complete the booking and what the trip microservice will then do is it'll then go and confirm the long-running action. So if I type a copy of that, run that one. So that's gone then and ran it and that's gone and closed the LRA and then the LRA has then closed all the completion callbacks for the various microservices that were registered with this long-running action. And you can see the aggregate booking, they're all in the status confirmed. So you can also do one with Cancel as well. You can do the first class too. Yes, you've got to go to get economy if it's available. And Simly, if you decide you want to abort the whole trip then you'd ask the trip controller to cancel the, cancel the whole trip and that would then just close the LRA and then the LRA would do all the right callbacks. So the trip manager, the trip manager, microservice doesn't have to cope with all the complexities of cancelling everything and shutting everything down because it's already registered all the logic with the coordinator. Effectively, in this environment, it's a REST-based system which is in JAXRS so they're just all the endpoints that it's registered. There is a second part of the demo which is running on Minishift. So Minishift is an open-shift environment that you can run locally. Let me make sure. Let me start the console. So I have already deployed these three microservices and also the coordinators into Minishift and this is showing them all running. So there's the flight coordinator, there's the hotel coordinator, LRA coordinator. So now we can interact with the open-shift environment with these services running in that environment. So I've pre-canned some of the commands. So this is going to the trip microservice so that's the endpoint in which it's lifting it, listening. So if I run that one, so with Kubernetes you have service-based URLs which then you have the actual pods that implement the services running behind that service so that's what that Minishift IP, et cetera, is doing. So that's created a booking ID. So the booking IDs I'm using are actually the IDs for the longer-in-the-actions so that's like you could have a mapping between the longer-in-the-actions, the booking IDs and the longer-in-the-actions, but for simplicity, booking IDs are the same as these longer-in-the-action IDs. So that's the booking ID that we've got back from starting the booking and then to finish the booking you would put a request to the trip service and then that will go and close the long-running-action that's running in the coordinator that I've also deployed to Minishift and that's showing them all confirmed. You can... two minutes. Another example would be to show that you can balance the coordinator so you can start that running. So what I've done there is I've started a booking going. The long-running-action is in progress and then what I can do is go to the coordinator. So he's a coordinator. So with OpenShift I can scale down. So if I scale that one down so when that scales down to zero that means there's no coordinator running environment so this is just going to show that there's some resilience in the system so the whole point of transactions and models is you have reliable guarantees so when things crash and things fail you can always come back up and continue where you left off so that's scaled down. So if I scale that back up again then once it's back up again I should still be able to complete that trip booking. So... So I'll balance the coordinator and then I can ask the trip microservice to complete the booking and there it has completed the booking even so it demonstrates that the coordinator does maintain state and it does remember how that is and which of the long running actions it's responsible for. OK. So it's good because that's where we are. We have a break between now and there's a break now between now and the next session I think it's about 25 minutes so if you want to ask him some questions he'll outline out of the session and we'll show you what this session is about. And if you take a look at the slides there's various links of where you can find out more information about what we're doing and a project which implements the transaction manager and presentations there as well and also the information about the microphone for our weekly hangouts we have. So that's the place you go if you want to contribute any input about what you might want to have changed. So for example like I said before we had some guys from China they're working on a SARG project and they did contribute changes to make sure that we can support things other than just rest-based systems. There's no time for questions. Right, offline questions then. There are enough time. There's no, is there? It just goes by like this. I don't feel bad. 35 minutes is quite a short time. There's normally 50 minutes.