 Yeah, thanks everyone absolutely ecstatic to be here. There's been some phenomenal talks So trying to follow them up is a little shuffle, but let's do this So today's topic is data DevOps and data science what works and what doesn't So just a little bit about what a ricto does we're really involved in the Q flow community and contributing since you know Oh dot for doing a lot of release management lead multiple clue for working groups and then internally we Also have products around cube flow and cube flow as a service so helping just people get more out of their platforms with open source Toolings such as cube flow The intro was really great. So I'm Chase Christiansen I'm a solutions architect at a ricto really my job is to you know work with MLEs as well as platform engineers and data scientists and they kind of listen to their pain points understand How we can help them with open source tools and and get them the outcomes. They want been through all the super the Kubernetes certification which means I know just enough to know that I know nothing about Kubernetes Right as you start to dive into those you learn about just the ocean of Kubernetes knowledge There is out there and I'm really really excited and spiritually aligned with this kind of trend of platform engineering Which is you know kind of DevOps for the purpose or with an outcome of building a platform and really helping people get the Bat most out of their work So I'm driving some MLOps initiatives and some of our open source strategies So I'm really aligned with the needs and Over you know and users being data scientists or MLE engineers and having been on the software engineering side and also understand I mean you know engage into more solution in an architectural discussion. So it's real fun So let's quick that's just for fun to get quick poll here. Who here would have aligned with being a data scientist? Who any data science in the audience? Okay, there's a few ML engineers, and I know these definitions are kind of fluffy. Okay data engineers Am I missing one straight developers? Okay, is anyone none of the above Okay, is anyone all of the above? No, there's a few here that are like reluctantly admitting it and they look really tired Those are the ones taking the most coffee. So totally understand MLOps is not demops And we kind of learned this the hard way and we actually see this a lot of end users And we think it's because the wires got crossed, you know the last year We've been hearing about MLOps and saying you know not a couple years right MLOps is about automating and testing your deployments of your software A model right a model is just some application you can serve data scientists out there training them You know they're doing wizard math on the board. They're figuring it out for you They're getting these data sets with the hard work of data engineers and then you just want to iterate faster, right? We've got this idea of hey if we iterate faster our models be great And then we say well listen we know about something that helps us iterate really quickly right DevOps And of course we know DevOps is different hand-wavy based on your organization But this concept of hey if we collaborate better if we understand and be mission oriented we can drive better outcomes So hey high-velocity we both align with this models or applications. So let's just start doing DevOps And it's not our fault right When we do things with MLOps through a DevOps lens we start to make these assumptions that may not always be true Right data scientists should write their own code data scientists should promote their own models Right they might be make a model push to prod they're good You know we throw a party their models perfect they buy all the right houses And then data scientists should package their own work via CI CD right we are starting to say data scientists We know that you work really hard understanding data sets and math And now we're gonna teach you get and we're gonna teach you get and you're gonna like it and sometimes that could be really difficult right and As you may have seen in person these These assumptions don't really work for data scientists right their work could take months and weeks and years not days Right if something goes wrong with a model or a new project is being presented looking at a data scientist and say alright You've got this assumption. You've got this hypothesis. Give me a prototype by next Tuesday They're gonna freak out right they're gonna be like what are you talking about? I don't even know if I can get the data right there's a lot of work They need to do they're required their models and how they do their work requires a lot of experimentation right domain knowledge and many many many Interations their models are not the full application a lot of times They may be serving a model or building into a model application that then using that logic to present some sort of business outcome and Many above these problems aren't just about serving the model right just because you've served a model and you can iterate on the model Doesn't mean that was the problem. In fact a lot of the talks today We've heard about listen. There's this new idea of maybe it's the data Maybe we need to be happy better about our data hygiene understanding how to source our data Because the model could be you know an open-source model or whatever and it is doing our job It's just your understanding of the domain or whatever is causing quality to to fall And this is the outcome right we're saying you need to understand our square DevOps And then there are circle and you're trying to ram it through and it causes a lot of tension And here's the real problem that we're trying to solve Data scientists need to pull data. They need to train models They need to validate models and they need to collaborate on model development, right? They're not just they're not the entire story right they want better ingredients They want better models data scientists also want to use common tooling and platforms to simplify this handoff process They want to give to the organization the work they've done, but they don't necessarily want to run these models Right they do not want to be in charge of you know being paged in real tonight because an imprint service went down Right that's not they just want the model to be built and they want to know about the quality of the model Which is kind of a different way of reporting things, right? And if we drill even deeper They want this process to be simple and they want to trust it without context switching Which is really hard because they just put on all these months weeks years to do these this work with this model And they want to trust that if they push this process that you as an ML engineer or a data engineer have given them That their model is then going to be put into production in a way that provides value for their business for their boss for owners or whatever To align with their goals and it has to be reliable and trusted Because data scientists is more than just models right data science is way more than just models and sometimes We're only just a small piece of the puzzle here, right? The implement each details and the tooling has all this background of hey Can we even solve this problem with data? Do we have the you know the right design and order is there an investment decision and then hey Can we even do this right and then of course the iteration of did we actually do it? So we had some users come up to us and they said listen We want to start to use these dev-op workflows And we said well, you know what if we build this if we build this amazing platform It's amazing tool set with all these cool open-source tools They've been hearing about at cube con for all these years It's gonna be great and the data scientists are gonna love us and they're gonna put us on their shoulders and Parade us around town So this is what we tried to build So we know data scientists like cube flow right they like cube flow. They like to be able to build pipelines They like to use continuous integration you can shift left into the data scientists world They can do all sorts of cool things we can get all sorts of little great artifacts that we can check into Kubernetes We also know that they like Jupiter notebooks, right? Which is why we start to look at cube flow because they can start to orchestrate things with Jupiter notebooks And we think hey if we have Jupiter notebooks if we have cube flow And we have something else to manage and schedule our compute we can help data scientists do their own build pipelines Which is why I use Kubernetes, right? You need a way to intelligently schedule things once you have these pipelines need to be able to create these jobs and move them Wherever you want the data science to be self-service And then you have to find a way to serve the model right once you have this model Are fact you find a way to serve this model and you know this is a whole another decision And they don't really want to dive too much into this they just want their model to be served So we give them canico right we give them a docker file. We say here's your docker file. Here's canico canico is great It's rootless. They can run in the cubicle pipeline just build things and go, right? and Then we use Argo for Argo for Argo workflows Argo just for delivery So you just update a manifest you can deploy things and move it across environments But there's a lot of little decisions in here that we had made for the data scientists We asked the data science to decide right how are you going to take your latest model, right? What does latest mean how do we version that and then how do we bring you on board with this? And it worked kind of right and I think we've all been in this situation where we built something really great And you know, hey, I've gotten successful container builds You know, I've done a sample light GBM model and served it with a container. What's wrong with you? But really we have just created all sorts of cognitive load and complexity Giving them a promise that hey, we're just giving you a platform. Hey, we're just giving you a build process So where did we go wrong? What we went wrong is actually the first our first reaction of data scientists doing this. This was not bad This was actually the best thing that could have happened to us, right? The data scientists looking at us and saying what you built does not work for me Probably saved us because we want this be able to scale beyond us We want to scale to all sorts of orgs doing ML ops, right? So the fact that we give them this over complex workflow and they push back was actually really healthy for us Because what we heard what we heard is listen the way we designed the systems and the way we have the interfaces within the system Is actually designing the tension and the ways that we collaborate within each other with our organization, right? This is like a loose interpretation of Conway's law, right? How we design our system and organize around ourselves around it is how the interactions that we're promoting the incentive to have these Interactions within ourselves our systems need to be flexible and transparent for us, right? They can't be over complex MLE's SRS platform teams They need to understand how these services are provided and they need to be simple to work with and it also needs to be flexible and lean and Transparent right if someone wants to run a new type of workload It shouldn't be a nightmare for them to do it and we should be able to be pluggable and let them do it We also learned that we need to listen to our data scientists way more, right? When you're developing things in a vacuum and trying to build a platform and not bringing in your data scientists You can run into a trouble and your first prototype might be you know kind of what we had which was the dumpster fire But that's not to say that there isn't a place for DevOps in data science, right? Because as you just heard we learned a lot of lessons we learned that hey We built some tools, right which took DevOps processes, right? Argo all these cool things had really smart people using DevOps processes or naming DevLock prop DevOps processes To make services to build really interesting solutions to help data scientists do what they want to do in an easier way We're actually helping them with their cognitive load and we can really respond to this quickly We can take a lot of different attempts We can make some wrong assumptions and we can continuously improve Using these kind of DevOps processes that we learn and that's how we work together to build a platform And we've done this we built some really great tools That have actually helped use these DevOps developer skill sets We have to help data scientists use their skill sets best in the Stefanos gonna dive into that So yeah, we want to build variables and we're trying to more or less talk about the philosophy and the standards behind certain design decisions And how do you find a way to solve them or enter them in the pipeline We'll be taking more specifically a look into how to do compile load-booting, pipeline, how to sell them in the notebook And we've seen how these kind of approaches are very well received by the community So, back in model service, if we are how the service pattern really helps us in building services that are easy to use And allows us not to reinvent the wheel and not to, you know, wait years with dollars and just building services from scratch So, going to service helps us building services that are users to know what they need to be Because at the end of the day, it has to provide a kind of evaluation on building models for data scientists And also a massive angle of dependency, runtime, virginity And a bunch of different inputs, protocols offered by a standard load of machine learning frameworks And that handle provisioning of computing that work with torsion and so on So, enter pacer Most of you sure we know about pacer and there was a nice talk about it today as well So it does go through very good with that What we're providing is a blue and white monoclonal store service It's a service with a very easy to write, young file, extra hours And it can even support generic packing back-end It has legal features like all the scale-a, intelligent routing to model models Pretty cross-processing, it's going to be able to be monitoring models There'll be able to be a service with big protection and many others And that advance deployments will be, it's like, canary allowed and many others And then, chill video library, where you're hoping to build You have to reuse common tooling that was built by the community And by very smart people who do the, is this enough, right? So, getting down to the bunch of young files or even, you know, a few orders of packing So, again, right? Platform with bundle pacer And at the end of the day, every organization has their own rules Assets amount of quality, integrates object stores Or a bunch of satellite services that you can interact with, one another And, eventually, all these things develop All surveys up to the end, users Because you have to take them into consideration You start building your models So, let's take a couple of examples Say, storing a model and publishing, publishing it to an object store Well, you need to start thinking about how you sell a model How to back it up so that you are sure that the case around that You will stop from using the word, you know, that's compatible And then, how do you manage to make sure, or versioning How do you make sure that the higher key you're building Your, you know, hence the word, model, versioning, large scale organization That you can go to that object store and they know how to use that So, that is a lot of complexity Since we have a certain disk, how familiar you can build Or any better organization of complexity Just build a bunch of, just provide a bunch of high-level ITR You may not So, now you reach the data scientist where they are Because they're giving them a very simple to use API And they're like, yeah, how do you make models of disk Or if that reminds of, you know, you can even translate this And maybe it's that reminds of iPhone catalog But then, after giving them one-to-one API Or API that they seem to have now Will be configuration tokens and coding from an object store They don't want your platform provided with support So, we actually, in our environment Analogs is not really about just a part of it It's about principles to machine learning It's more about, you know, the application And services that are paid for So, we've done it So, again, this is something that we've proposed That is the way to go It is, to now we want to sort of Use data science, every capability And how do you use an SRE And that will be useful for your next build And for us, well, this is an open question And this is something that we'd like to discuss After a full day of talks about analogs And we'll see how much this way of thinking Resonates the community to better support You'll find that they're good So, yeah, see you today