 Thank you everyone for coming to my presentation. My topic today is distributed workflow for microservice style application. So microservice architecture is adopted for breaking down the big software application into loose coupling service module. It provides a lot of good things. Sorry about that. So the microservice architecture provides a lot of good things such as failure, isolation, scalability, and it's easy for the development. But it also introduce some new complexity. Suppose you're going to implement your business logic on microservice design. So how are you going to manage your message, traveling across the different modules, and how you're going to coordinate your task execution on different services, and how you're going to handle your errors. So to make your life easier, I think a workflow pattern will give you the capability to implement your business service logic over loose coupling service modules. So before we start, I'm going to say something about myself. My name is Rin Chen and I was a former network engineer, and now I work as a software engineer in Nomada. A company delivered the integrated solution on multi-cloud application management, as well as Kubernetes-based container management. Also, Nomada is a Kubernetes certificate service provider. So my presentation is to introduce the distributed workflow pattern and its usage in microservice style application using Nomada OSS workflow library. So here's the outline. So first, I will briefly introduce what is microservice and what is workflow, and what is the challenge about the workflow management on microservices. Then I will introduce you the Nomada OSS workflow and try to do the demo, and also we will talk a little about other solutions. So what is microservice architecture? Basically, in microservice architecture, service model worked independent and isolated. In most of the case, we deploy those service module on different containers in Docker world, and the different service module will communicate each other through let's say REST API. Then what is workflow? So workflow basically is the coordinate of a sequence of tasks. Let's take an example. If you want to do a shopping workflow, so your first you're going to place the order, and then you're going to pay the order, and then you're going to get your product shipped. So there are three tasks in this workflow, and each tasks are processed by different service module and they are sequential. So we need a workflow to coordinate to those task execution. This is monolithic web application. So what about you want to scale your application in a distributed environment? That means you have multiple instance for your web application. Accordingly, you have multiple instance for your workflow. Then you will need a scheduling module to coordinate to those workflow instance. Now we come to the microservices. As we mentioned before, microservices module are independent and isolated. So the challenges are how are we going to execute task asynchronous over the distributed environment across multiple services. At the same times, we need to manage the task dependency, and also we need to handle the complex logic, such as what if one of the task is failed and you want to keep going in your workflow, or if one of the tasks failed, you want to stop all the workflow, kind of things. NAMALA OSS workflow solve those problems. It is an open-source library written in Java based on Apache Zookeeper which is the well-known reliable and highly available coordinator. We also use the Curator as the client, Zookeeper client. It is lightweight and easy to use. So here list some main features of the workflow library. It is capable of manage the task relationship. The task relationship here mainly means the task dependency, and it could schedule task in the distributed environment, and the task type could be customized. Also, it gives a good support for fault tolerant. For example, maybe anything will go around during your task execution. Say your service is restarting, or your Zookeeper cluster change state, then the task execution will be resumed. There are some key components in the workflow library. First is the workflow manager. The workflow manager will register for a specific task type when the service initialized. So the workflow manager is responsible to submit your task and give notification, say there's a new task node there. The scheduler will manage the task node during the task execution. It will ensure that the task node will not be executed twice because it's a distributed system. For the task queue here is a wrapper of Zookeeper node where we put our task node in. The task executor here is just an executor thread. You can have multiple task executor thread in your service. Let's go deeper to take a look at the task execution module. So when the workflow manager submits a task, it will create a new task node in the task queue. This is task type specific. The consumer will listen to those task queue. If there's a new node come in, they will be notified and then one of the task executor. Say we have multiple task executor instance. So one of the executor thread will pick up the task and execute it. So the scheduler here will ensure that no other task executor pick up the same task here, and also the scheduler here will make sure that if the task has a sub-task it will extract the sub-task and then execute that. So the whole executive model is decentralized. The workflow manager instance and the task executor could be on the same service, but it's not necessary. It works well in the distributed system, and the task could be executed asynchronous. So here the diagram mainly show how we manage the task dependency. The task dependency is represented by a directed acing click graph. So those tasks has no dependency, like task one, task two, and task three can be executed concurrently. But for task four and task five, take example here, should be wait to the one, two, three finished executing that it will get started. So we can handle the concurrent and the sequential task execution here. Here I give some example code for how to build a workflow. You might need a workflow manager builder to build the workflow and pass by some related parameters like in which namespace. This namespace is mainly related to the zookeeper namespace. So the namespace to where you want to put your task in, and so the auto-cleaner here is to say that when the task execution be completed, you want to automatically clean up those task nodes from your task queue. So how many times you want to wait after the completion, and how often you want to run your auto-cleaner you could define here. Also, here we can add in task executor. The task executor means how many concurrent task executor thread you want for this task type. Yeah. So, and also here is an example code for how to write the executor. Take example here is I just get a logger here to print that the task could be executing. So, this is one executor. When you're adding multiple executor, it will execute the task at the same time. Okay. Now, I'm going to show you some demo to see how our workflow library works in our production. So, this is the Mata web applications. I'm going to create a new environment here, use a shopme blueprint. Okay. There's a lot of service here in this application. So, each service is a micro-service. So, what I want to demonstrate here is I want to show you how we manage the task dependency in our workflow. If you're looking to the blueprint to see the shopme and go to see the service here, you can see the dependency. So, for catalog, it has no dependence and the customer also. But for deals, it depends on catalog service and customer service. The loyalty here is the similar things. So, what I'm going to do here is try to update the image of all those tasks, all those service. Okay. Let's say there's a service fail here. Let's just remove that. Yeah. This is how the Mata manage the micro-service. So, I'm going to do the environment update to all the service here. I choose an image. I choose a new image for all those service. Let's say what happened. So, underneath for each service, the image update is an independent task. But the whole workflow including all the task here, and it will take care of the dependency of the task. For what we have seen before, the catalog and the customers has no dependency. So, those tasks will be executed first. For the task like deals or I think his loyalty orders are dependent on the previous task. So, it will be executed later after the previous task be completed. Yeah. You can see that for those tasks has no dependency, it is executing concurrently, and for those tasks has dependency, we'll wait for the previous task finished executed, then it starts executing. So, yeah, this is what we call a workflow. So, basically it consists of multiple tasks, and the tasks over different micro-services, and we just manage to do it here. Yeah. Okay. So, this is finished. Let's go back to the slides. What I just show is how we use our workflow use the Demata workflow library to manage the workflow in our productions. Of course, there's other great solutions in the industry, such as Netflix conductor which is the orchestrator but also provides a feature that you can define your task, execution using a JSON-DSL. Other like AWS simple workflows, it's a Cloud application. So, you can use that in your Cloud applications across multiple machines. These both are great solutions and integration solutions. It's just we start with our ZooKeeper and the Curator framework, and we just want to write a library which is lightweighted and easy to use, easy to plug in so we develop our own, yeah. So, that's pretty much about it. So, thank you for coming. So, I just wanted to see if you guys have any questions about that. Yes. From our experience, a single task workflow always do, and for what I showed before, it's like wrapped tasks. Wrapped tasks, it's a task chain, like you can do dozen of tasks in your workflow. But yes, I haven't tested, I mean, in our current production, we haven't faced the issues that we have hundreds of tasks there. So, yeah, maybe we will try later, but technically, I think it supports. Any other questions? Yes. Our production, you mean production or workflow library. There's two different things because our workflow library is the open source library. There's a GitHub link on the slide. But our production is, yeah, it's a production, business production. Yeah. I'm not so sure about that. But I think if you're using, if you already use, because we use ZooKeeper and Curator, sometimes it's, to be honest, it's difficult to monitor and troubleshooting them. But so, if you already have your ZooKeeper cluster runs, maybe you can have a try. Okay. So, by the way, our company have a boost downstairs. So, if you guys have any interests or you want to know more about our workflow library or our production, you're very welcome to come to our booth. Okay. Thank you.