 So much for the kind intro. Thank you so much. Well, um, yep, I'm JP Zibelich. I'm the CTO and founder of pipe hit. We are a control plane for our good workflows. Um, I'm currently based in Paris, France, where in my free time, I love to butcher the French language. This, why would you, uh, this, this week, we had a very fragmented landscape of orchestrators. We had already a few concerts on platforms that focused on, like, doing model training where you see deals and stuff like that. Yeah. Thank you, well. Yeah. So when we set out to build pipe kit, um, it was kind of roughly the like 2017, 2018 era. Um, and we spoke to a bunch of data scientists and engineers about their problems. And they all mentioned like, Hey, we're running into problems with dependency management and scaling. And the obvious solution to us was like, well, why not just use Docker and Kubernetes? That's like what those things do. Um, and they said that the tooling wasn't there. Um, the dev experience wasn't mature enough. And so we got to work on trying to figure out how to bridge that gap. We then found Argo workflows as being the workflow orchestrator that seemed to be winning in the cloud native, like workflow orchestrator wars and said, like, all right, let's dive in a little bit there. That led us to find that many people were building out these workflow orchestration platforms or these data science platforms internally and spending a lot of resources doing it. So he said, Hey, why not like use our expertise and start building that out as a service? So there's a set of common requirements that both of our platforms had. Obviously we all wanted to use like Kubernetes for our compute layer. Both teams agreed that Argo workflows was the orchestrator of choice that we wanted to go with because it's built using like Kubernetes native primitives. You're interacting with like pods and all the same things that you'd be interacting with. Like if you're just like writing, you know, any sort of like application definition. We also needed to both scheduled jobs across multiple clusters from a single control plane for Bloomberg's team having like a consistent rest API was super important. And for us on piped it in addition to a rest API, our customers are asking for like tight integrations with their Git providers, such as GitLab, GitHub, and also a CLI that ran on top of what they were already using for Argo workflows. Going along with that, our customers needed strong role based access control so they could say what users have access to work with. Our customers were asking for like tight integrations with their Git providers, such as GitLab, GitHub, and also a CLI that ran on top of what they were already using for Argo workflows. Going along with that, our customers needed strong role based access control so they could say what users have access to what resources on what cluster, which expands like an extra dimension than what you can currently get if you're running like a single cluster set up with Argo workflows. And then one other thing that was kind of interesting is we both wanted to set up some sort of workflow template versioning within like a vanilla Argo workflows setup, you have access to workflow templates, which is how you like reuse code, but you only get like one version. And so we wanted something similar to what we do with like Docker images where you get like, hey, I've got like my ID service or whatever and I've got like a, you know, half a million like tags that are associated with that. We wanted to replicate that for like easy rollbacks with Argo workflows templates. We also wanted to render a library of cluster workflows templates to give our user a standard way to do things such as HTTP requests or then a Slack notification. Concerning further, we also value stability and security and Argo is so flexible about whatever you want, while we don't really want our user to run whatever they want. So we only allow cluster images that's built with field types, and then we want our workflow templates, their names to follow a semantic versioning standard. So our user will be able to name their workflow template something like my auto template that's 1.0.0. This way it's easy to control the variation and it can be widely used in the game mode. And lastly, workflow templates must be promoted to product environment. So that means we don't allow workflow templates environment to be created directly. They have to exist in the environment first, and then maybe the recovery itself is very important. So if in an event of one visitor being wiped out, there's something hurting or whatever, we don't want to use certain templates and anything going on there. So, but the Argo frameworks will, you know, fit all the deal because those frameworks are counter-specific. So we use an internal source assistant Chrome scheduler to do our Chrome scheduler. And then lastly, we want to do an auto support blueprint. We don't want to enter requirements for cell 8. So we'll do it from cell 3 to 6, but I don't use 3 to 4. Great. So we have some pipe kit specific requirements and approaches that we needed to tackle when we were setting out to build our platform. So they can be filtered into like three broad areas. The first was ease of use. The second was security. And then there's some just kind of like additional, I don't know, let's call them like junk drawer features. Like, you know, that like drawer in your house where you're like, I don't know where I'm going to fit like all the other stuff. We'll put them in that category. So when it came to, you know, just like using the platform, again, our customers had like three things that they wanted. One was like a CLI where they could schedule workflows on different clusters from that single spot. Essentially something that would be like a drop in replacement for the Argo workflows CLI, which performs wonderfully for like a single cluster setup. Second, they wanted like an HTTP API. So if they were using like some sort of like web hook or something like that, they could integrate into that nicely, specify like a workflow definition, specify like the cluster name and send it that way. And then lastly, something that we get asked about all the time are these like Git provider connections. So connections with like GitHub, GitLab, BitBucket, things that are going to be, you know, like a dedicated GitHub app or GitLab app, whatever the BitBucket equivalent is so that we can read those like web hook events, like pull down the code that might be like hosted, you know, run things and then update whatever the like checks are on each respective platform with a good, you know, like check if like the workflow succeeded and acts if it failed so on and so forth. And that just does a lot generally to like make the dev experience like much smoother and easier to use from the security standpoint. Initially, when we set out to build the product, we were thinking like, hey, people want like, you know, a hosted version of the platform, right, right. Well, that turned out to be wrong. Like people wanted to bring their own clusters, especially if you're running like data processing. You know, you don't want to trust that to like a third party. You want to say like, hey, I have X number of Kubernetes clusters under management. I want to bring these clusters. And additionally, I want to make it so that these don't have to be exposed to the public internet. So we were a little limited in like, say, setting up like endpoints on each cluster. We had to figure out what is like some sort of like poll based or like what we went with was like a queuing based system so that, you know, we could easily say like, hey, you know, we're going to limit like the security footprint here. And then lastly, like similarly was like bringing your own logging provider. So again, initially when we set out, we thought like, hey, people will trust us with their logs. That turned out not to be the case. People want to bring their own like logging back ends and that's something that we support as well. Then circling back in on the workflow template setup, whereas Bloomberg and company was able to go with that Kyberno based solution, you know, using like a very opinionated setup and a mutating webhook. That isn't something that scales for like multiple like companies that are using like the single platform. We have to be like very flexible in what sort of like workflow template names and tags that we allow. Whereas like Bloomberg was very intent on like, hey, we need to limit you strictly to that semantic versioning like we as a vendor can't be opinionated on what like an end cut company like wants to do. So what we did instead was say like we're going to rely on the get provider to give that back end there and say like those are where the workflow templates are stored. And then the get tags, whenever like a user cuts a get tag for that repo or that like workflow template file, that'll serve as like the tag which the Argo workflows like runtime will then have access to. Hopefully it'll make sense when we get to the diagrams. Anyway. Thank you. Yeah, I would resolve faster and send a buyer to the cluster. This is what an order number one folks kicking. So basically our camera. Why would you go check if the workflow is using those. And along with some data. And then a. And. Thank you so much. So our architecture follows like somewhat similar patterns, but there are some restrictions that we run into. Since again, we're not just serving like one company, but several companies. So we'll break it out in terms of layer. So the top layer is how our customers are interacting with the control plane. Again, I mentioned that CLI HTTP API and then the get provider like back ends. Next, each one of those is going to send what is essentially like an HTTP request to our hosted control plane. Now with Bloomberg's approach, you saw that they use like Postgres as a source of truth for state across their multiple clusters. And we use what is our hosted control plane, which is essentially just a Postgres database, but with API layers on top of it. Right. Like if you're connecting like multiple different people to like, you know, a similar database or the same database, you need to have good primitives for authentication and things of that sort. So it wouldn't be safe to just say, like, hey, we're going to open up like a Postgres database and, you know, let everyone like directly connect to it. So the pipe gate control plan serves as like a like central API layer for storing state for all of our different customers. Now next we've got this message to layer. I'm going to briefly jump into that and get into what like the individual clusters look like. So whenever a customer like registers their cluster, they have Argo workflows installed on that cluster, but they also have the pipe gate agent. And that does a lot of the things that you saw will cover previously such as like garbage collection and things like that. But it also reads in our case from these like message queues that get spun up one per cluster. I think now we've done it with like multiple message queues per cluster. Right now we're using SQS as our message queue provider of choice. But we're thinking about moving to Redis to have some other like nice primitives such as like cancelling in-flight messages and things of that sort. So each cluster then, you know, is reading these message queues and saying or listening to events that are emitted by the pipe gate control plan. And once we get that event that comes in, that tells us like, hey, we need to schedule like an Argo workflow on like this namespace. And on this cluster or actually never mind, it's in this namespace, right? Since each message queue is dedicated per cluster. So if a cluster says like, hey, you know, this is the message comes through, it knows I'm going to schedule like this workflow on this cluster. And throughout the lifecycle of that, it's sending HTTP requests back to our like hosted control plan to update the state of what the workflow is doing. And so that each one of our users can have insights into what's happening across their entire fleet of clusters. All right. Now, let's bring it back to some like contrast and comparisons between our two solutions. So the first problem that we both tackled was like multiple or multi cluster scheduling across different sets of clusters. And a pipe kit, we also went with an agent based approach. The difference being that we are relying a bit more heavily on message queues, given that we don't have as much like direct API control and control over the infrastructure that Bloomberg does. We're also providing that like host to control plan with, you know, again, a dedicated API layer providing that source of truth instead of the direct access to the Postgres database. And a pipe kit, as much as I would love to force everyone to use some verb because I think it's, you know, awesome. We just can't be as opinionated as like a Bloomberg platform team can. So we set things up again to just read workflow templates from get tags. And that gives people the flexibility to, you know, make like tag whatever and just kind of send it, which, again, you know, I would like to be a little bit more opinionated. On that one, but that's not always realistic for everyone's needs. All right, let's get into a demo. So I'm going to go ahead and show importing a cluster in the pipe kit UI. We're going to go ahead and create this cluster within the pipe kit organization. And we've already got one cluster running, but I'm going to show you how to install like pipe kit on my Docker desktop. Cool. So we went ahead and created this like cluster manifest. I'm going to go within our terminal here and apply the YAML that we generated. This is going to spin up our pipe kit agent. Currently we already have like a functioning Argo workflows instance running that you can see here, right? We've got the server and the workflow controller. And we've got a 30 second, you know, readiness check just to make sure that everything's working. Looks like we created our message queue successfully. And in seven seconds, this guy should be live. Yeah, all right, let's go be left. Cool. Yep, it's live now. All right, so now that this is live, we can demonstrate how to submit workflows to multiple clusters. We have a cluster that's hosted remotely. That's called like a runner cluster. And so first I'll show you like what YAML we're going to send to it. This is just some like dummy test YAML. You can find this in like the open source Argo workflows setup. It's going to log, you know, a whale to the console. So first we're going to submit this to our like remote cluster and we can see that we've spun up a new workflow. It is currently running. We're going to get the logs of that workflow available to us. And then we can see on our local like Docker desktop that nothing's there since right, we submitted it to that remote cluster. But next, you know, when we submit another workflow, we should in this case be able to see like, hey, this workflow is running on Docker desktop. So we went ahead and submitted it or not yet. I'm going ahead and submitting it here and we're specifying JP Docker desktop, right? That's our new cluster. And boom, I'm going to refresh the page here and we've got another DAG that's been declared here. We can see like this is the state of the workflow that's being run on the second cluster that Docker desktop that we just imported. And so we have the logs coming in live and we can see like the global state of the workflow. And if we go to our like Kubernetes visualizer, we can see on our Docker desktop that again, this workflow is running. The pods are getting spun up and we have access to information about these pods, right? So we can see on the first cluster, the workflow has been completed. And if I go back to the Docker desktop cluster, we are completed as well. I think that should be it. I think do we have time for Q&A and seeing the 10 minutes in the back? Yeah, we probably have time for like one quick question. So get your hands up fast and then we'll do it. Yeah, we'll go over that. Yeah, I love the approach by the way. That's really cool to see like, you know, how like a company implements it as well as a vendor like alongside and the compare and contrast. I don't think I've ever seen a talk like that. That was awesome. Great presentation. Do you plan to integrate it with Argo events? Like integrate with Argo events. Was that the question? Yeah. Yeah. As a vendor, I don't know, right? That's something that, you know, we got to like just pull our customers figure out like how tight of an integration they want, or if they want the integration and, you know, go from there. How about Charlotte Bloomberg? Thank you. How good. Great. Alright, I'm not going to keep everyone because I think it's time for a break now. But obviously, you know, go and refresh yourselves and then come back. We have three fantastic talks lined up afterwards. So yeah, thank you again. Let's have a round of applause for Will and JP.