 Good afternoon, everyone. I'm Andy Bonham, Senior Distinguished Engineer, Architect at Capital One. Today we're going to talk about open-source BPM and also do a comparison of some of the common BPM products. First a little bit about Capital One, we are a bank that's built like a tech company. For about a decade, we've been investing in becoming a tech company, bringing our engineering talent in-house and building our own software solutions. Our tech team is about 11,000 people and we're all in on the cloud. All of our capabilities are built on AWS. We also have a pretty advanced DevOps pipeline that automates a lot of the manual tasks that developers typically have to do and allows us to use a microservices architecture and allows our engineers to really focus on innovating and building new capabilities at the top of the tech stack. We're also big into open-source. In 2014 we made our first contribution as a company to open-source and a declaration that we're open-source first. Since then we've released about 40 open-source projects. One of the most popular ones is Hygeo, which is a DevOps dashboard that's used by a number of companies across the country. You can see a couple others listed there. We've also invested a lot of time over the years too in building the right culture and governance to have open-source and highly regulated banking industry. If you're interested to learn more about Capital One or looking for a new opportunity, definitely stop by our booth. We are hiring. There's also a link here to Capital One Tech. It's a good good site for a lot of blogs that are tech leaders right and you can check out Capital One careers for the openings and also follow us on Twitter. Alright so today what we're going to do is pretty much start from ground zero assuming that you don't the audience doesn't know anything about BPM. Introduce you to some of the key concepts and then gradually get more advanced and in the end the goal is for you to walk away today with a good understanding of what open-source BPM is and also some of the distinctions between some of the common products that are out there today. Real quick a show of hands who has used or is familiar with an open-source BPM product. Alright it's about half. Cool some of this might be repeat for some folks other folks might be might be a value so we'll start from from the beginning. So what is workflow? It's often a overloaded term. It can mean four or five different things depending on the context that it's used in. The way I try to think of it is it in the end it's really about individual or discrete steps that are happening in a business process and that can take the shape of three or four different forms where it could be human workflow where it's steps that you need a human to perform. You know investigate this case or route this application. It could be more orchestration based or system orchestration where maybe you have five APIs you need to call in a certain order and it's there really isn't any human interaction. Then it could be a combination of both or maybe it's system orchestration and then when you hit a certain condition then you have human workflow that that executes. So when you're talking about workflow it's good to understand the context in which the term is used because it often is used differently. So now we know what workflow is let's separate it into two different buckets simple workflow and complex workflow. Let's first start with the basics of you know what are some of the characteristics of simple workflow and at its core task management or a to-do list is one of the common characteristics that you'll see. Users need to know you know what are the tasks that they need to do and also maintain the state. What's in progress what's not started what's done and then notify either them or other other folks when certain phases are changed either through a text message or email and then normally simple workflow involves maybe only one team or division it's not multiple groups so it's kind of consolidated into a single group. Moving on to complex workflow often then you start and see interaction across multiple groups multiple divisions across a company and then the workflows start to get more complex you have multiple levels you have exception paths you have callbacks where things need a route back you can also then start and see forms of case management where it's more investigative or research based activities that aren't 100% predictable they may not follow one flow but they might follow two or three in an ad hoc or on on demand nature so that adds an element of complexity. You can also have integration with rules so if then logic where you may have policies that you need to execute it could be many rules based on on the use case but it could be integration with either decision tables or rules engines and then as we mentioned a little while ago system orchestration where you may need a in addition to doing human workflow also orchestrate API calls in a particular order. So what is a BPM product what problems do they solve? You know at its core BPM stands for business process management and you'll often hear the term you'll hear several terms you'll hear BPMS which is business process management system you may also hear I BPMS which is an intelligent BPMS management system and at the end really what these are is they provide a capability to model a process and also execute it. There tends to be a separate suite of products that might just focus on the modeling aspect but the BPM products focus on the modeling and also the execution they also take it a step further and provide integration capabilities with plugins and adapters and really what the problem in my mind that they address is when it's easier to build or code a business process in a visual model than in code that's when these really start to end up in their sweet spot is enabling visual solution and modeling of a use case or a problem and there's a there's a ton of open source products out there there's at least 15 or 20 you can see some of them in the word cloud there so today we'll focus on like three or four of them that are Apache based first before we get into the actual open source products let's talk a little bit about some of the common notation the industry standard notation that is affiliated with a lot of these one of the first ones is known as BPMM which stands for business process modeling notation and this was a spec created by the object modeling group and basically was very detailed spec that landed on agreement on certain icons and activities of how diagrams are built so you can see here activities events gateways these all have very specific icons and standards that were created and what this did is it enabled a consistent way for modeling a process so now these BPMM diagrams can be portable where you can take them and use them across multiple systems that support the specification as long as the systems don't add their own proprietary modeling on top of it it also enables collaboration between business and tech folks where you may have a business process analyst that can model the process and then your your tech folks can take that and implement it in a BPMM or BPM product so it enables a lot of collaboration and standardization another notation that you'll hear of is decision modeling notation or DMN this one focuses on decisions and rules similar to BPMM it has a flow that's followed and then it gets to a point where a decision table like in the example is executed where if certain conditions are true then a result will be returned it is compatible with BPMM so they can be used together in a in a process and then moving on to the third notation again this one was by the object modeling group case management modeling notation that you see there in the middle this one focuses on cases again that investigative or research based dynamic activities that are not really as predictable as a flow and usually in the open-source BPM landscape the products will use one or more of these often it's all three or either at least two with BPMM and DMN and we'll talk a little bit about that in distinction between the products in terms of which ones they support in a little bit but now that we we understand what workflow is what BPMM is some of the standard notation what are some of the alternatives because like anything in tech you know one size does not fit all there may be some cases where a BPM product may be overkill some of the alternatives you can explore are state machines that may not be as detailed with the visual workflow AWS step functions is one they have a workflow studio that is a visual modeler but does not use the BPMM notation others are X-State IO which is a JSON based workflow and Netflix has a conductor project that can be used for orchestration so these are all their alternatives again if maybe you don't need all the capabilities of a BPMM product but need some of the orchestration and workflow capabilities so when do you use a BPMM product this was a chart that I put together to explain to folks and again it kind of goes back to the idea that when when you reach that point where to be able to build your process it's easier to manage it in a visual model that's usually a good sign that you want to use a BPMM product if it's overkill to do that then other alternatives may be a better fit and usually that ends up in two dimensions high integration and high complexity of the business process those are your sweet spots of where a BPMM product and its capabilities can really help solve your problem if you're low on the integration or low on complexity a BPMM product might be overkill and it's really up to you to evaluate your use case and to see if you need the capabilities that are provided and you can see in here the bullets they're sprinkled in basically from what we covered earlier concepts of simple workflow which are in that lower left quadrant and elements of the more complex workflow or in the outer the outer boxes but you really have to sit down for you know and ask what's the problem you're trying to solve and you know make sure you're selecting the right solution the other thing you need to be aware of is the monolith it's easy to build a monolith application in BPMM products some of the products themselves the way they're deployed are monolithic there's a number that have been moving towards more of a microservice-based cloud native architecture but this is one thing you want to you want to be aware of and really the idea is you want to break things into microservices build smaller pieces that are more module so that way you can scale those independently instead of having to scale the whole monolith so definitely something you want to keep in mind as you're you're making the decision what what to use and how to design it some of the scalability things you'd want to consider you know if you do have a process you can think about breaking it up into sub-processes and deploying those independently there's also a you have different options in terms of deploying some of the BPM engines one option is to use an end process or embedded mode where the engine itself actually runs under the JVM same JVM as your application or you could run the engine as a standalone deployment where it's running as its own service and your application and integrates with it through an API now both of those have their pros and cons the end process or embedded approach can have better performance because it's part of part of your application there's not any additional network calls being made but it does introduce coupling right it's tightly coupling your process engine to your application and it also makes scalability a little more challenging right you can't just scale the engine or the application since they're embedded typically have to scale them both the standalone deployment approach that approach provides an abstraction layer so it doesn't necessarily matter what technology the application is built in as long as it can call an API it can integrate with the standalone deployment you can also scale the standalone deployment separately since it is its own deployment but one downside is you do have additional network calls through the API so again those are pros and cons that you have to decide and make tradeoffs on depending on your your use case the other one that I often have folks think about is the third one around what things you should do in the BPM product versus what things you should do outside kind of gets excuse me kind of gets back to the monolith conversation throw everything in the kitchen sink inside a BPM product you want to be selective and build the things in the BPM product that the BPM product is good at like what are its strong points and that typically is workflow or orchestration I've seen teams kind of throw everything in there and then it leads to other problems later on so you want to be very declarative about what is in the BPM what you're going to build in the BPM product and what should stay outside because a lot of the BPM products have ability to integrate with external your interfaces or services which can help you integrate it if you do keep it externally all right so now we'll move on to the open source BPM part where we'll start and dive into some of the products I did a little bit of research just to flag some of the key milestones in the history of open source BPM there's a lot more to this there's you know at least 15 20 plus products I kind of kept it to you know for that spawned off of JBPM at some point and that's JBPM activity command up and also flowable but if you look at this it JBPM the first version came out in 2003 and wasn't until about 2010 when activity launched which was another open source product you'll notice that the first version of BPM in came out in 2004 followed by the second version in 2011 commanda forked off of activity back in 2013 and one interesting pattern you'll see is a lot of these products tend to fork off of others and they may reuse some parts of the architecture but completely do completely redo other parts then along you know the the other two notations CMMN and DMN we see came out in 2014 2015 and then the next open source products came out that forked off of active activity known as flowable in 2017 then there was a period of time where different products supported CMMN and others decided not to and then more recently in the past two years Kajito is a new refactor of the architecture with JBPM that's more geared towards making it cloud native and microservice-based and command also had one of their new releases this year so one of the things that that stays the same is that things are always changing in this landscape so you really have to kind of see what's out there at any point in time as you're evaluating different different products we're going to do a deeper dive now of four of the products and again these at one point in time we're all a patchy based and one interesting graph I went out there and pulled a Google trend graph this is a graph that just shows you over time different the popularity of different keyword searches in Google and you have to take this with a grain of a salt because you know often other words get baked in command in JBPM are fairly unique words activity and flowable had some other things you know get baked in there like concrete and yogurt so you have to filter this out but it's interesting over time because it starts to show just you know the interest in what folks are searching on JBPM for a while was the top search keyword followed by there was a spike when activity came out and then activity kind of led for a while and then around 2017 communda has been the more popular search term in BPMN so just just an interesting graph this doesn't correlate that I'm aware of to the number of you know who's using it this is more just searching searches that folks are doing all the terms and in Google so what we'll do next is starting to get into the detail of each of the products we'll look at it from three dimensions the open source model this is very important to understand because each BPM product has a different open source model and it can be confusing to understand so you want to make sure you understand what you're getting out of community and also enterprise in case you're trying to make that decision between one or the other we'll also look at the capability set across some of the capabilities you see here like a rules engine, a modeler, different deployment modes also cloud managed services or past SAS type type capabilities then we'll take a look at the community activity the contributors of the commits how active is the project so let's dive right in and I'm not going to read each of these line by line but kind of hit the highlights and then we have a side-by-side comparison at the end we'll start with JVPM this is one of the ones that's been around the longest around 19 years just Java based it has both community and enterprise versions their open source model is fairly straightforward everything that's in community is in the enterprise and vice versa typically their enterprise is about two branches behind their community because they want the stable release that's in community and fairly active every about every three weeks they're doing releases some of the big things about the capability set JVPM tends to have one of the larger capability sets they have their own rules engine known as drills they also support all three of the industry notations and one thing that the JVPM project has been focused focusing on going back to our timeline graph since about 2020 is in this new Kajito project and this is where they're building the refactoring of the architecture JVPM you know some of the complaints over time has been it's really big you know big thing to deploy and manage and this Kajito project is basically refactoring it and making it more cloud native and microservice based they're also looking to add some capabilities around reactive messaging and serverless workflow and right now they're they're taking kind of a dual strategy of continuing the traditional project while also providing Kajito as another alternative moving on to Kamunda Kamunda is about nine years old they also have a community and enterprise edition their open source model is a little different where they have their their decision engine known as ZB and then also their desktop modeler available in the community version there's a slight nuance around source available and open source there but for the most part it is is open source then there's a set of capabilities that come with the enterprise and aren't open source but you can use and QA things that help manage the operational aspect of it Kamunda supports BP MN and DMN but not CMM and that was an intentional decision where based on community feedback a few years ago a lot of the community found CMM in is being overly complex and a lot of the things you could do in BPM in a simpler way so based off of community feedback they decided to drop support for that and continue on some of the other key things about Kamunda is it it supports standalone and remote deployments in version 8 and intentionally remove the embedded deployment model and part of that was around scaling and coupling constraints where they found with the standalone remote deployment it gives you that abstraction layer and it's easy to easier to scale so that again was another intentional decision that they found helped help the community provide better solutions the other big thing with Kamunda is they do have a SAS offering known as Kamunda cloud it's based off of GCP so that is that is out there if that is something you're looking for moving on to flowable flowable again forked off of activity about five years ago it is they do have a community enterprise version and they have a slightly different open source model compared to the other two where what they do is they provide pieces of all the different projects but then when you go to enterprise you get additional capabilities on top of that so that's where it's really important to understand what are the pieces that you need for the problem you're trying to solve and understand if they're in the enterprise version or the open source version they do support all three industry notations their modeler is a little different where it is a plug-in into Eclipse and one thing that was interesting there's a blog out there I have links to a lot of these in the notes where they recently showed how you can run the flowable engine in a AWS Lambda and a serverless approach which is pretty cool moving on to activity it's 12 about 12 years old they also have a community and enterprise a vision enterprise solution as the others their model is very similar to flowable where they provide the pieces in the community and then there's additional pieces in parts you get with the enterprise they are very spring boot cloud friendly that's one of the common pieces of feedback I get as I talk to engineers and developers that work with it it's very easy to integrate with spring boot they do have a web modeler and also have an eclipse plug-in similar to commando they have a cloud offering it's a platform as a service known as al fresco cloud that you can use if if you're interested in those so you can start and see some of the nuances and the differences between the different products you know now let's talk a little bit about the community right it's really important with an open source project that you have an active community you want folks contributing to it making commits so you're not the only one that's that's doing right you want a community that you can go to for help and and help enhance it and keep the project going on so I pulled some of the latest commits for the four and you can see here from the graphs the activity of the commits and contributions over the last nine years and when you look at this you have to kind of adjust it for the scale because 20 for JBPM is a different scale than 20 for commando but when I looked at this it you know what I came came away from is commando and flowable there to be the two most active recently in the past least in the past year for commits JBPM was probably in third with commits they were a little lower but you can see from the graph they were fairly consistent and then activity seemed to have periods of being really quiet and had a spike so I don't know if that maybe was due to the fork of flowable but just interesting kind of interesting stuff there and then the other thing when you look at these two if you go back in time for when say flowable forked off activity if you look at like the first half of the graph it's pretty much the same it's because they forked off of each other so there's nuances in there that you can see but it's always good to check this out on the insights tab on GitHub when you're you know trying to figure out if you want to use it definitely want to active community this chart is pretty much just summarizes what I'd already hit the highlights on across the four again this is my kind of perspective after doing some research JBPM had the largest capability set which could be a pro or a con based on the problem you're trying to solve you know there's differences between the capabilities around embedded mode and modler and support for a CMMN and again a lot of differences between the open source models of each of these so anyway what I would say is if you're trying to figure out which one to use like understand how they're different and then try to find the one that matches your company's principles and best practices to help you help you move forward put up here a couple blogs we've written over the years we actually did this lasted this comparison about six years ago and when I was putting this together it's kind of amazed at how much has changed since then but a lot of other interesting articles out here if you're interested these are all in that Capital One tech site a bunch of others that my colleagues have written do stop by the Capital One booth if you're interested we also have a networking event later tonight that you can learn more about so happy to chat more about this and that I think will take any questions sure so the question was what is the what are the use cases where BPM can be a fit for like CICD project management it's good question it can I mean there's no limits to what it can be applied to that fit into the workflow realm I've seen it used in CICD processes there's a wide spectrum of problems it can solve you know I would think of it more of you know from the dimensions of human workflow and also process orchestration are two places where it can help call center yep absolutely yeah that's so the question is what are the web forms for each of the products yeah those are if you think of if we go back see here that have forms called out I don't think I did if we if you think of like data entry for workflow where say a person gets a task that they need to complete and they need to either put in some notes or add in some other things a lot of these tools will automatically generate a UI based off the data attributes some of them are fairly I mean it's not a fancy UI you know they're very basic but it could be a quick way to generate something that you can get out into production for for data entry a lot of use cases I've seen as teams will go out and build their own UI and you know more modern technologies like you know angular you know forms of you know JavaScript or React view and basically interact with the engine through an API cool so the question is is are any of these more cloud native like Kubernetes based when I looked at when I did the recent research commando and flowable have built out support for Kubernetes along with activity so that that seems to be a common a common thread that they're moving towards where you can deploy it Jbpm in their core G to project is also moving towards Kubernetes I know in the traditional project they had a way where you could deploy it in a Docker container but it was literally the whole application so I don't I don't know that would call that Cal cloud native you know you want it you want to split up into you know services and have multiple Docker containers but I think they're evolving over time we use a host of several of these and to the other over the question was what products we use a capital and always forget to repeat the question sorry it so we use we use a mix of these and it's across a range of use cases to the other question you know there's there's a wide spectrum of use cases it can apply to and I've seen them used in kind of both to me actually all three dimensions of human work flow process orchestration and a combination of both yeah great question so the question is in capital one what has been our experience with building roles in a reusable way that is a great question and over over time you know I think we've we've learned along the way you know like everyone else where you do it the wrong way and you fail and you learn you know how to do it the right way you know things as simple as naming conventions having a rule repository having a way to discover things like it's one of the foundational things right to be able to reuse anything you have to be able to find what's already there if it's not easy to find folks are gonna use it they're gonna build something else and then duplicate it so you you often some of those basic things you have to solve for and then things like drools do have a rules repository that integrates with get so you can store your rules and get it on the back end any other questions good question so the question is the capital one do we have any of these products that are used as like an internal service for other divisions to use or is it more embedded and I've seen both where in some cases teams will embed it because their use case was a little simpler and they didn't need to serve like multiple applications but then we have others where maybe they needed to serve an entire division so they used more the standalone approach and you know leverage the API so multiple applications could could reuse the same you know same deployment good question so any is there any product that's better with the shareable or the embedded you know I think some of the experience we've had with definitely activity our spring boot developers that was a plus they felt it was really easy to integrate with spring boots and but the embedded can be done with you know with JVPM or you know the older versions of commando it was version 8 that believe it moved to just the remote or standalone but that's that get to your question kind of gets back to that point of how modular you know is the application like is it one big library or is it broken out into smaller libraries and all you need is like one library to include so each of those is a little different on in terms of how modular the components are they do have a web modeler to my understanding I need to come into the folks to check me on this but I think it's only available in their SAS deployment but that was my understanding any other questions think of anything feel free to stop by our booth and we'll be hanging out there if not thank you for your time in the discussion