 Good morning. Welcome to KubeCon and CloudNative.com and to this Dapper Maintainers talk. We'll introduce ourselves. My name's Mark Fussell. I'm a Dapper Maintainer, and I'm also the CEO of Diagrid. And I'm Chris Gillum, software architect at Microsoft, so we're focusing on serverless cloud platforms. And today we kind of want to dive into a little bit about what Dapper is, just get a little bit of the background. We're going to spend quite a lot of time, and Chris is particularly on workflows, which is probably one of the most exciting things that we've been building into Dapper in the last few releases. And then I'll kind of follow up with a little bit about talking about some of the things in that latest release, particularly around the outbox pattern. And then we're going to finish up with sort of the road ahead and where the actual project is going and sort of dive into sort of looking beyond the future. You know, what is Dapper? Dapper is effectively a set of APIs that allow developers to build distributed applications, to build cloud-native applications. It brings productivity for you to build across distributed platforms like Kubernetes. In essence, Dapper is a set of APIs that codify best practice patterns that save you time building things like request reply or pub sub-event messaging or long-running stateful or workflow applications. And by combining these together and coming at it from any language or any runtime of your favorite, you can build applications that run on a set of VMs across Kubernetes. It's not bound to communities and platform, but it's about productivity for developers. So when you're building applications, Dapper APIs allow you to distinctly build them more productively fast and then half the time you'd expect. In fact, we did a great state of Dapper survey a few months back and it showed that Dapper itself saves developers 30% of the time getting their applications into production. Now, another way to think about this is that Dapper is a set of patterns that you can use from each one of your runtimes. So you can take one of your favorite languages, you can take your favorite framework inside of it all, and you can think of them as patterns that you see typically codified like saga workflows or request reply or long-running event driven applications. And Dapper alone is very powerful, but you can be used alongside other APIs and frameworks that you use. It doesn't have to be solely used and embraced. It's kind of a combination of effect that you can bring together with other frameworks. So when you look at this as well, you can look at it through the eyes. Well, I was using microservices.io as a place to show microservices patterns. We kind of layered the common use patterns on top of that, and so you can see that even from a microservices sense you can bring these consistent patterns to your development. Now, that is it from a developer perspective, but what does it look like from a platform perspective? One of the key questions as well that we kind of need to answer ourselves is how is it that Dapper fits in with hosting platforms that you use? So when you look at three very common hosting platforms, you get the less opinionated Kubernetes platforms, very powerful, but you have to think about a lot how to deploy and run to them, and then you get the very opinionated functions platforms. And to some degree, Dapper sort of embraces all those different levels. But one of the common questions is what's missing from the conversation about how is it that developers using those platforms talk to underlying infrastructure? And what we see commonly is that developers bring in a particular SDK and they bind tightly to particular infrastructure libraries and sort of tightly couple those two. So one of the key attributes that Dapper does is it allows you to have a separation of those concerns. It allows you to have your underlying infrastructure through the API contracts and also through the idea of a component model so you can swap out the underlying infrastructure yet make your code consistent. This is very relevant to an audience here where you're thinking about platforms and how they have contracts that fit with the application world in terms of the application developers building on top of them. So in a particular case, I'm very powerful with Dapper as you can build applications locally and you can also then move them to the cloud and all you have to do is swap out this underlying component model. Or you may have built an application that's even outside the cloud but be able to say there I'm using particular underlying technologies that I went to from Message Broker or State Store and without changing my code, I can move them into a cloud and swap out the State Store I was using for a cloud store or cloud Message Broker. So this all gets surfaced effectively in what Dapper is called the component model. You know, for each one of the... behind each one of the APIs is a component model. So for example, behind the State API, there's a component model for State Stores about how you store State in a key value store. Or behind the Message Broker, the publishing API, there's an API to plug in different Message Brokers. And this portability of code prevents you from having lock-in, it provides flexibility, and, you know, time and time again, we have conversations with organizations that go, I wish I hadn't tightly coupled my code together. Now, Dapper does layer a lot of functionalities on top of this. It's not just purely an integration play. So, for example, in the State Management Place where you have non-transactional State Stores, Dapper provides transactional semantics for multi-rights across the door. Or Dapper provides security at the topic level for security messaging that's layered on top of all those Message Brokers. So it's very powerful runtime in its own right providing a lot of features around patterns that layer on top of this component model. But this combination of component models and patterns giving you portability and flexibility, you know, translates effectively into, you know, how is it you build a distributed application? Well, here I might have an application that uses PubSub to coordinate through a various State Store that sends messages to another server that's sending messages out to Twilio, and then it may do a Direct Invoke message to another service here that saves some State or talks to another cloud service. And when you look at the design of this application, binding them all together effectively is a set of Dapper APIs. And those APIs make it consistent. It allows you to have a contract that you talk about how you build your applications. And the flexibility and portability of that is incredibly important. In fact, the more we spend time with customers, the more we spend time with them. In fact, the more we spend time with customers, the more we see that this contract of having an API that keeps the portability of your code and all the rich features that Dapper has so that you can build applications quickly becomes very important to them. But what's also critical is across all these APIs is you get consistent observability, security and reliability. So if my calls fail to service invocation, they can retry or my PubSub does retry. I do end-to-end security across all of this. So you get security out of the box. You get observability out of the box. You get reliability out of the box. And by the time you put all of this together and you think about this, what really Dapper is is an integrated set of APIs that all work very well together and provide this seamless connectivity to pretty much give you 80% of the whole app you want. And with the introduction of workflow, which is in so many applications today, we have a business logic that has to run, that communicates with other systems, that does service invocation calls, that talks to external systems. You can find that, you know, you have a very powerful suite of APIs that runs on many different platforms, that communicates with... runs on many different cloud services you can hook up to, and you can see your productivity about building applications. So with that, I am going to switch over and let Chris dive into workflow and all the amazing things that have been happening there. All right, thanks a lot, Mark. By talking about what is a workflow? And I think this is an important question because depending on who you ask, you might get a very different answer. So we'll start out with the definition that we used in the Dapper workflow project, which is a sequence of software-defined tasks or activities that are performed to accomplish a specific goal or objective. So obviously that's a little generic, but sort of gives you an idea of how we see what workflows are. Diving in a little bit deeper, let's talk about some of the ideal characteristics that we think workflows should have. One is that they're stateful and durable. Right? We want them to be able to execute to completion regardless of infrastructure failures. Another is that they could be either short-lived or they could be long-running. Right? Workflow might run in just a few seconds or it might take weeks to complete. Maybe never ends at all. We believe that workflows should be virtual, meaning that if you were running a workflow in one VM and we need to load balance that thing, we should be able to do that. To unload it from memory, reload it to another node transparently without sort of impacting the availability of that workflow. Also, addressable workflows should have some sort of an ID such that we can send messages to that workflow to help it move along if it needs to. As well as clear life cycle semantics. Right? So these aren't just state machines. They have life cycle semantics such as pending, running, suspended, completed, failed, terminated. You get the idea. So given that, we came up with a set of features that we wanted to surface in DAPR workflow. Here's some of the main ones. One being activities. These are the basic units of work that workflows are responsible for scheduling. They have at least once execution guarantees and they do sort of the core processing for making out about network calls, those sorts of things that workflows often do. Durable timers. The ability to schedule arbitrarily long delays in workflow execution. Again, this could be a few seconds. This could be a month long delay. The workflow shouldn't care. Child workflows. So the ability to sort of break down a larger workflow into smaller, easy to understand subflows. This idea of external events. Workflows should be able to interact with the external world around it. Not just be completely isolated to its own set of activities. So for example, being able to take in an input. If there's like a human in the loop process that needs to be automated. And then lastly, retry policies. So if you have an activity that doesn't work and that it fails, just the ability to be able to retry that using either a static sort of declarative retry policy or even a code based retry policy. Let's jump into the code piece in a little bit here. So with DAPR workflow, you were actually writing workflows in code. And this is an important distinction that we have in DAPR workflow versus many other traditional workflow engines. Today we support .NET, Python and Java with Go and Node.js support coming very soon. But the important thing to note here is that the code that you see, and I apologize for being so small, but this is ordinary code just implemented as a simple function. Right? This is not something like Airflow where you might be using code to construct a DAG. Rather, we're just using idiomatic code in the language of your choice to actually define the workflow execution. In fact, you can even attach debuggers to this while the workflow is running to actually step through and see what's happening. And this allows you to use variables, loops, local variables, try catch, sort of exception handling, all with the guarantees that I sort of mentioned earlier about being able to run sort of reliably and durably, being able to move it from one VM to the other, so on and so forth. So if you've ever heard of, say, temporal workflow or cadence workflow, or even Azure Durable functions, it's effectively the same idea that we've sort of taken and put into the DAPR project to make available in a cloud agnostic way. So the way that this basically works behind the scenes is there are two pieces. There is the DAPR workflow engine which runs in the DAPR sidecar, and then there is your workflow code which runs within your application container. Now, the workflow engine, what it does is it takes care of all of the state management. It takes care of the sort of failure handling, retry, basically all the scheduling of actions for you, whereas the code that you write is purely focused on, you know, what is the business logic that I'm trying to implement, right? What are my tasks? What order do they need to run in? Those sorts of things. In fact, if you dive in a little bit more closely to sort of see this protocol, there's actually this two-way sort of GRPC streaming protocol that we use between the DAPR sidecar and your workflow application where the sidecar basically tells you, alright, please start running workflow X or please start running activity Y and then your code goes ahead and it runs that particular function or at least that step of the function and then it'll send back a result to the sidecar and then the sidecar then goes and saves it into the state store. Basically creating sort of an execution log so that if there is a failure, we know exactly how to rebuild that workflow and get it back into the state that it was in before. So with that, I would love to jump into an actual demo. This is an order processing demo but I do want to highlight that DAPR workflow in addition to being able to do things like business workflows is also very good for other scenarios such as infrastructure provisioning massively scalable sort of batch processing type jobs and just a variety of other workflow scenarios but we'll focus on sort of this more business order processing one for now. We have five micro services in this process. We're using the DAPR state store. We're using PubSub as well as the DAPR service invocation building block which allows the workflow basically to communicate to all these different services either directly using service invocation or indirectly sort of using PubSub via Redis. So I'm going to go ahead and switch over now to visual studio code where I have an app. Hopefully this is visible enough. I can zoom in just a little bit here but over on the left hand side you can see I've got four services, inventory notifications, order processing, payments and shipping. I also have a DAPR.yaml file which you can sort of think of as definition of our app for running it locally. Now normally in production you would run this in Kubernetes but DAPR has a lot of great tooling that allows you to also run your apps locally on the machine. The main app that we're going to focus on is the order processing app. Now this is all written in Python and if we dive into this, this is the app that's going to define our actual workflow logic and I want to spend a little bit of time on this. So we have a bunch of business classes which we have here which are totally custom and then we have a set of functions. We have a process order workflow which is our workflow function and we have a bunch of activities which are the pieces of code that the workflow is basically going to orchestrate in a reliable way. Now the workflow takes in two arguments there is a workflow context and then there is some input which is some a business entity that you define within your app in this case an order. And if we look into this function a little bit one interesting thing that you will notice is that especially if you're a Python developer you'll realize that this is a generator function and so what we're doing is making this context object which is given to you when you schedule this workflow we're able to use it to do various actions like calling an activity which is going to do some work in this case you pass in what is a pointer to the function that executes that activity and the result of this call is going to be a task and if you yield this task like we do here in this generator function what's going to happen is we're going to basically tell the dapper runtime which is running in that side car please execute this task that I wanted to execute in this case the notify one now if we take a look at the notify task it's rather simple it just takes the dapper client and invokes the publish event API to publish the message to a pub sub broker in this case it's just a redis pub sub broker but that's effectively what it's going to do now this notify activity it might run on the local machine it might run on a completely different machine basically depending on the load balancing characteristics that we need at the time but the idea is that this workflow can schedule these activities as it makes progress another thing that I'll mention to when we do this yield it's going to pause the workflow wait for this activity that it calls to complete and once it completes the generator moves to the next step which is in this case calling another activity reserve inventory that's sort of the next step that we do in this workflow when you call these activities and you yield them you can also get the output of the activity so for example when we reserve inventory we get some output back saying was it successful we can write some if conditionals to say hey was it successful if not send a notification that it failed we can exit the workflow at any time to complete it using a return statement again just using idiomatic code that you're used to to author all the steps within the workflow and again each time we do a step we do this yield which tells the sidecar hey please go execute that work that I asked you to and which point we also save our state so that if there were to be some sort of an infrastructure failure we're able to still resume this workflow load it back into memory and resume from the last yield point that we had so just quickly to walk through the last steps we'll get to this part in a minute but we sort of submit the payment after we've reserved the inventory and then we submit the order to shipping as one of the last steps here but you'll notice too that we wrap this in a try accept because if that's shipping operation if it fails for some reason well we've already submitted the payment so we need to compensate for that which is why inside of this accept block we have another activity where we can call refund payment to make sure that if for some reason that we're not able to complete the shipping we're going to refund whatever that payment was and so this is an important sort of guarantee that we offer that we're going to run through your workflow code the presence of failures and so if there is some sort of application level issue that prevents shipping you know we can have compensation for that which is often a very important feature and then once that's all done we do a final notification to say that we're done and then just to quickly oh and one other thing that I think is pretty interesting too so I've showed you so far just orchestrating a bunch of steps let's say we want to add a human approval in the loop as well let's say that if an order is over a thousand dollars somebody needs to manually approve this thing well workflow has tools for that too so one of the operations one of the tasks that you can schedule in a workflow is waiting for an external event you define what that external event is in this case we're calling it approval and let's so we're going to pause the workflow and say we're not going to continue until somebody approves this we receive the approval event but at the same time we also don't want to wait forever for somebody to approve this let's say we want to bound it to like 24 hours so we have another task which is a create timer task you can set any duration to this in this case 24 hours and we're not and basically we're going to wrap those two tasks using this when any API to say when either the approval task or the timeout expiration task complete go ahead and resume the workflow and return back whichever of those tasks was the winner so below that we can say if the winner was the timeout expiration then you know we can notify the hey we we failed to receive an approval for this and we go ahead and complete the workflow as a canceled order effectively otherwise you know check to make sure that we got the approval that we expect and then continue running the workflow so anyways that's that's sort of an interesting feature that we also have as part of workflows there's a few more but we'll focus just on that for now and then lastly I just want to show really quick we have a few API this is just a flask a flask app as well where we allow you to take adapted workflow client you can schedule new workflows say which one you want to schedule give it an input you can give it an instance ID so that you can reference it later which we're going to do so enough talking let's actually run this so I'm going to do dapper run and what that's going to do is it's going to take all these five microservice processes that I've defined within my app and it's going to start them up on my local machine and what I'm also going to do while that's starting up is I'm going to open up on the bottom half here we have our notification service which is the subscriber to all those pub sub notifications that the workflow is going to be publishing so anyways we see down here that it says connection open and we're going to go ahead and start a workflow so hopefully I can do this in a way that's somewhat readable and I go to the top here and let's go ahead and submit an order this is going to be a slightly expensive order sometimes this is a little funny okay there we go so we're going to submit this order and the order was received and immediately you can start seeing that the other service which is subscribing to the notifications is now getting those notifications that we received the order we reserved the inventory we're waiting for the approval because it exceeded our threshold so that means we need to do the manual approval now one thing that I also want to show you here which I'll go ahead and maximize this for that let's see how do I get back my here sorry okay another thing I want to show you is structure failures so I'm going to actually control see this and now I have just shut down all of my processes in my microservice sort of simulating like a data center outage or something like that so we can see in our UI that the connection is closed we've lost connectivity everything is down at some point though you know our ops team will come in they're going to get it all back and running again but you know what happened to our workflow right it was in the middle of running waiting for approval will it be okay and so the connection is open so we know that our systems are up and running again what I'm going to go ahead and do now is try to send a request that is going to complete that approval process so I'm going to go ahead and send that we received it and what we should see which we do is that the order was indeed approved the payment was processed successfully and it looks like there was an error submitting to the shipping I think that's because maybe I did it a little bit too fast but anyways the important takeaway here is that the workflow was able to continue where it left off without sort of needing to redo any of the previous steps which we can sort of see by checking on this notification service here so that's a very important capability of the service last thing that I want to show you really quickly if I can find my mouse pointer we have distributed tracing that we support with workflows as well so if you need to go and sort of debug and see what happened you can do that we have a nice display here in Zipkin which shows you first of all the full span of the workflow itself how long did it took in this case 79 seconds and you can actually see of all those activities that it's scheduled how long did they run for what was their status and so on and so forth and even you can sort of dig in and see some other details like it failed doing the shipping step but anyways sort of that end to end distributed tracing transparency that you would hope to get with a workflow system that's calling services a whole set of different microservices so anyways that's where I wanted to show Mark I think I'll hand it back to you it's pretty incredible stuff workflow is we're going to switch laptops here just a moment so we're just going to jump over here okay hopefully it should spring to life so yeah that's workflow I mean you can see how powerful it is especially combined with the other DAPA APIs let's dive in a little bit more here because I think another incredible capability we added into DAPA runtime is another pattern if you think of workflows around Saga we put also the outbox pattern what is the outbox pattern the outbox pattern is a combination of doing a transaction around a state save and also a message send so for example say you are saving in the orders or account information and then you want to tell another service to send an email you don't want to send an email saying your account has been created and then save it and you don't want to save it and tell a person that their email wasn't sent so you'd like those to be bound together under a single transaction that's exactly what the outbox pattern does so for example in this case I can save my order into the database or my account information and then send a pub some message that's sent to another emailing service and that tells my customer whoever it is that I have now created your account and it's ready to go and you want them to be consistent it turns out it's pretty hard to do in an easy way so we're going to show how on any one of the 15 or 20 state stalls that we have inside a dapper runtime you can simply combine any one of the pubs or brokers with any one of the state stalls to do the outbox pattern between them all and I'm going to do it between Redis pub sub for my message broker and I'm going to do it for my SQL both running my local machine to show you how you can put these two together let me just switch over here into VS Studio so VS Code so let me maximize this okay so I've got two services here one's an order processor service which simply creates new orders inside and it's going to save them into a SQL store locally and all it does is it does uses the dapper clients inside here to execute a state transaction into my local SQL store and it just saves a number of orders in fact I just do two orders inside this and then I have an order notification service all it's doing is it's listening on the pub sub broker to orders and it just publishes and I receive a notification from my pub sub broker in this particular case my state store I'm saving into if I return here you'll see I'm saving into my dapper store name here which is my SQL store and these component model as we talked about in the beginning of this talk is defined here as a SQL state store and I kind of have my connection string how to talk to it all but in order to take the message of the outbox pattern I simply have to add two metadata properties into my component manifest I simply have to say the order topic that I want to send on through my pub sub broker and I want to say the name of my pub sub broker that I'm going to send it to so that means any time I save a state into this state store and at the same time I want to send out a message through my broker that combination will happen through the outbox pattern so that's all you have to do and you just simply have to add these two here different types of state stores that we have and be able to take advantage of inside your dapper run your dapper project so let me put this together I too also have taken advantage of using a dapper run command I'm going to show you first though if I look into my state store I don't have anything saved inside this so this is looking at my local mysql state store so now what I'm going to do is I'm going to simply run and I've got the dapper run command inside here combined into two microservices that are run together so if I run this you'll see that it'll run the order processor it'll create an order here it'll save it into my local state store and then you'll see that notifications get sent and actually I'm going to just check here that I put this into the right sql state store and then the notifications will get received inside my pub sub broker which I don't think to have got run in this particular case so let me try it one more time inside here so these are the notes we've received okay and then you see that when I run this you'll see the order notification received through my pub sub broker combining the two with my save so there's an incredible incredible feature that you can simply do this on any one of the brokers and if you go back to my state store inside here you'll see now if I run the query inside here here's my two orders that were created and saved and if I go and look in my redis actually look into my redis actually look into my redis one which I don't have a viewer inside here you'll see that the message that gets sent to my redis broker so incredibly powerful and these are the sorts of things that we're doing inside the DAPA project we're continually innovating to make sure that we bring very consumable easy to use patterns to developers so that you can spend your time building great business code and not having to implement common patterns like this that you should find and make them available for wherever you come from today whatever language whatever framework that you're deciding to choose so where's DAPA going and where's the roadmap of where we're looking for well we release DAPA every three months we release it four times a year every quarter we just did the 112 release about a month ago that contained things like the outbox pattern and then we have the 13 and the 14 release that will be coming early next year and sometime in spring of the year after that and there we're focusing a lot on getting things stable around workflow and bulk pub sub which is in our pub sub broker APIs we're looking at how we're going to introduce a scheduled API how do I run cron jobs on a regular basis so I want this thing to happen every day or every hour it turns out to be incredibly useful to be able to have an API that schedules things on a regular basis an amazing feature that's coming is the hot reloading of components today when you change a component you have to restart your DAPA sidecar to pick up any of those definitions we're having it so that if you change a component manifest and it'll be automatically reloaded for you without your DAPA seek sidecar starting up which in itself is a huge feature which has been long asked for so now you can keep your systems running and without deploying new components inside more another deploying model is you can run DAPA as a sidecar today that runs on the Kubernetes world per pod but there are also circumstances where you want to run it on a per deployment side that means any number of instances of a particular application can have a single deployment for a DAPA runtime for them so that just allows for different deployment models which is often asked for especially when there are some restrictions in your testing environment or maybe some of your deployment environments and then finally we do a lot make sure that SDKs have consistency between them and also having the idea that you can have DAPA running remotely that you can attach to over from your SDK so those are some of the immediate things as many many more other features coming into those releases but as we look beyond that there's a lot of APIs that we still take through to stabilization that we introduced over the last year around distributed lock and crypto APIs we're heavily into security we've done amazing work today to give application identity through SPIFI to each one of the applications but we want to take and do more there in terms of deeper SPIFI integration and also integration with cert manager so you can bring in external search from external search search authorities we are always looking at improving the local developer experience you saw us for multi-app run today that allows you to run multiple services on the same machine how do we do things like local component validation more types of local components and then generally inside the project we're very fanatical about because we do multi deployment across multi clouds and testing on multiple environments we focus a lot on testing not only of the deployment models but also perf end-to-end testing and making sure that we really have a stable, resilient system around them all and then I think one of the most exciting areas we've had a lot of asks for is state storage and how to improve the types of patterns that it has you see today that we have key value store as a type of state storage but as we look towards other things that we can bring inside there document store type, blob storage and even SQL storage as general purpose APIs so that you can have that abstraction layer over all types of storage not just sort of key value storage so now you can have that component model over blob storage of different clouds for example that I think is something that will really open up the different types of distributed stateful applications you're going to build and then beyond that we are fanatical about getting DAPA integrated with other things inside the ecosystem so we've worked really, really, really hard with other projects and if you are a contributor or a maintainer of another project that you think has a synergy with DAPA we'd love to hear from you for example we've done a lot with test containers and v-cluster and open function and critics and in progress as I mentioned more integration with SPIFI we're a certain manager but bringing K-native functions together with DAPA is an initiative we're working on and generally a lot of people have come to us and said you know DAPA APIs are becoming so unique, so specific and so well adopted we'd love to see the general open API specification around it all so that you may even have other implementations of them so please come and talk to us about how we can engage with you inside the other ecosystems how we can get DAPA integrated as part of all of this you know I'll leave you with the fact that DAPA is about making developers happy you know making you enjoy what you do not focus on the plumbing don't reinvent the pattern the pattern's been done for you build your applications use the patterns that DAPA gives for you and you know we'd love for you to come and join our 3000 contributors or 7000 discord community you can find a ton of other information inside here in terms of how to get started how to contribute a community called twice a month that we often look for having demos and deep engagement with the community and I'll just leave you with go off to our repo where if you want to get a community supported digital badge to show your excitement about the project scan this put it inside your GitHub repo it kind of shows how you support the project so with that thank you very much today