 Well, so yeah, welcome everybody to this talk. We're going to be diving into as the title suggests secure Python ML we're going to be looking at this from the lens of You know bad practice IE major security vulnerabilities that we can find throughout the end-to-end ML life cycle and also how to avoid them There's going to be a lot on this talk, but fortunately there's practical resources We're going to be delving into a hands-on demo so you can find the slides at the top right corner here So bit.ly secure ML. So let's dive straight into it a little bit about myself. My name is Alejandro I'm engineering director at Selen Technologies. We're a open core machine learning deployment and monitoring Startup so we've developed one of the most popular machine learning deployment frameworks in Kubernetes Almost a chief science at the Institute for Ethical AI where we focus on contributing To frameworks that ensure the responsible design development and operation of machine learning systems And I'm also a governing council member at large at the ACM So yeah, as I mentioned, this is a very practical talk So we are going to have resources in this repository which you will be able to find So this is the Jupyter notebook that we will be using throughout So if you do want to dive deeper into any of the areas you can try it yourself So yeah, and if you find any improvements, please do add a PR So let's take a step back and understand the motivations and the approach that we're going to take for this topic The topic of machine learning security So one of the key interesting things is that the security challenges in the DevOps and traditional software space Are very Actively explored. There's a lot of resources around them However, the key challenges in machine learning itself are still being explored So there's a lot of content around how to use machine learning in security things like spam detection malware detection but best practices of security in machine learning at least from from what what we have found is not is not as defined and For that what we want to make sure is that this talk is more of a call to action For us as practitioners to to continue exploring the best practices The way that we're gonna approach this is in a similar way to how it's approached in just general security Basically with the principle that it's impossible to make systems unhackable, right? But it's possible to mitigate on desired outcomes through introducing best practices And one thing that we have to remember is that even though the solutions will be largely technical In nature, they you know, they're always is going to be relying in humans and process, right when it comes to security You know, you may create a system that is like, you know, highly robust with minimal vulnerabilities But you know that the humans are still going to be involved there And there can still be a potential for social engineering that that can open loop holes So we're gonna be also talking about this concept called ML sec ops, which is now a growing new buzzword Which basically is the the intersection of DevOps sec ops and ML ops is taking You know the extension of DevOps with security But machine learning as a first-class citizen and specifically the infrastructure that that is enabled To make sure that that it's enforced, right that you have best practices on the security life cycle And you know, this is not a very general talk We're gonna dive specifically in the security side, but let's take a step back and also remember why production machine learning is so hard Right to think about this in an intuitive way We're bringing in all of the challenges that we face in the traditional software world But we're adding and sprinkling the machine learning on top, right? The challenges that you have when dealing with You know specialized hardware and the scheduling of the specialized hardware, whether it's GPUs TPUs Large amount of memory, etc. You also have complex data flows, right? You have not just a single machine learning model, but you have machine learning systems and you have Components that may affect other components down the stream or up the stream Similarly, there's dependencies of the data as it flows through the system and then that actually boils down into the next part Which is reproducibility of components, right? There is a challenge to make sure that whenever you want to rerun a specific, you know Inference request you may need to make sure that all of the components are atomic and are reproducible You know whether it's today or or in a couple of weeks and finally is the compliance requirements and the burden that it brings For the practitioners that are very use use case specific now when it comes to the security side We've actually we're actually going to be able to leverage a lot of the resources that have been created in the general software space So similar to a lot of the machine learning engineering best practices that are being introduced in data science We are also going to be able to take that approach in the security world and one of the you know Great resources is this resource called the OWASP top 10 This is basically a report that comes out every year that highlights some of the open web application security, you know project Highlights the top 10 most the top 10 highest vulnerabilities in the web space so you can see things like you know broken access control cryptographic failures Injection so remember this list just because by the end of this talk We're going to have a list similar to this but specific to machine learning So again, you know this talk is not not specifically about machine learning deployment So we're not going to be delving into into the details of that But we are going to be carrying out the the training packaging productionization and an inference of machine learning model There's going to be a little bit of a hand wavy, but you know if you are interested on the details You know, there's other another talk at a previous PyCon and pie data that you can check out You know in your own time So what we're going to be doing is we're going to be training a machine learning model And we're going to be packaging that machine learning model and deploying it So in this case is going to be a scikit-learn model We're going to be converting it into a fully fledged microservice that is going to support, you know rest GRPC and Kafka calls But you know the key thing is we're going to be using frameworks that allows to do this in this case We're going to be using this framework called ML server for the runtime So it's built on fast API and seldom core for the orchestration. That's what's going to be happening in in the background So let's get started So that's basically the notebook that that I was mentioning earlier What we're going to be doing first is we're going to be using, you know, scikit-learn We're going to be using the hello world of machine learning, which is the iris Classifier and the iris data set. So we're going to be training a simple logistic regression model And from that, you know, we can see that we've trained it, you know, very simple And we now have a machine learning model that we can use for inference. So if we pass an input, it provides us with the prediction So what we normally do now, I guess in a production end-to-end machine learning lifecycle We want to persist this machine learning model, right? So we're going to be using our handy pickle or in this case job lip to actually dump that binary And if actually let's have a look at what's inside of that binary just out of curiosity, right? So so so if we if we just yeah, I guess let's start the kernel Unless we run that again, and let's have a look inside of the pickle So inside of the pickle we actually can see some simple things that we would expect, right? So in this case, it is the module that you're expected to use This is the scikit-learn Logistic module the logistic regression class and then the parameters that you want to use to initialize that pickle when you load it From memory, right? So that's basically what you're expected to do So so far so good So now we can actually take this pickle put it in a remote bucket, right? So that we can actually productionize it in this case We're going to be using Minayo which can be s3 or Google bucket or something like that And then we're going to be able to deploy it into our local Kubernetes cluster So this is basically a step that I was mentioning very hand-wavy at this point But you know just for the for the sake of the token security. We're gonna have to yeah I guess skip some steps. So now we can actually see that this artifact that we that we you know Uploaded into a bucket on the the folder FML artifacts slash safe We're gonna be able to see that now we have a microservice running and this microservice similar to how we consume that locally We can now consume it in a remote way So we send a prediction which in this case is this input and then the inference response is basically, you know The one hot vector with the second class as being the prediction, right? So we've deployed a model, right? So so this is the stage where you know, we've done our job We're happy we can go to the pub But the reality is is that the life cycle of the model really begins once it's trained and once it's generating value and You know in other talks that I link we talk about other things like monitoring But in this case, let's talk about security and when it comes to actually Security considerations if we look at the steps that we carried out, you know, this is the traditional, you know ML pipeline, right? So data cleansing Future engineering then model training, you know, several iterations and then once you're ready persist the model and then deploy it And then monitor it. So we ask the question if you know looking at this end-to-end machine learning life cycle Where are the areas that you know, you would imagine are susceptible for potential security vulnerabilities, right? So if we have a think those are the ones highlighted in red, right? So every single stage of our machine learning life cycle can be Exposed to a security vulnerability, right? And this is something that we have to be conscious about when when going through this talk is that There will have to be processes that you can introduce but also Making them proportionate to the risk that is involved, right? If it's only one data scientist the amount of like, you know overhead and automation would be less than if it's a team of You know, 20 data scientists with three machine learning engineers with one DevOps engineer that is like, you know Producing models at scale So now let's actually look at each of those stages of the machine learning life cycle and see how each stage can be exploited so we're going to be looking at you know, some potential vulnerabilities at the loading of the models of the artifacts access to the model and you know potential issues in that front Issues with dependencies and supply chain management Code vulnerabilities model runtime images and then some honorable mentions on infrastructure So let's start with the first part. So in in the crowd here who has used a pickle raise your hand Okay, so for the video is about 90 percent have have used pickle So we all love pickles, right? We use them on a day-to-day basis. Should we use them? I don't know. Maybe maybe we shouldn't use them as as ubiquitously as we are So let's have a look at some of the challenges when when when it comes to to pickles themselves So let's take that pickle that we dumped that binary that we dumped and let's load it back into memory We can see that it still works, right? If we run a prediction, it still it still works But now if we actually look at what's happening and how Python interacts with pickles The way that Python interacts is through the redot reduce function And in this case what we can do is we can inject our own reduced function to tell Python how we want It to handle with this pickle, right? In this case what we can do is we can just make it run Return the package OS that system and this command which is like a base 64 string Which is going to be just taking all of the environment and putting it into a file called pawn dot text But this could be anything else, right? It could be pulling the Kubernetes secrets sending them over an email You can run any, you know command that you want to run So let's actually put that and let's inject it into our class now What we can do is we can dump that unsafe model that model that we are now calling unsafe And then when we actually look at the pickle, it's a little bit different, right? Instead of seeing the scikit-learn module and all of the things that we were expecting We can see that system module and then this command which basically says take the environment and put it in a pawn dot text file Right so on aware of this, you know, we are deploying our model We can see that now our models that are deployed So now we actually have our safe and our unsafe model and we look at the inside of the of the container itself We can see that that pawn dot text file was created So that code is running and if we now actually try to load that pickle locally We can see that the pawn dot text is created locally, right? So basically when loading this pickle we're executing this, you know arbitrary Potentially malicious code which you know Really we shouldn't be doing that So so now you have a bit of an intuition and you know fortunate unfortunately, you know But also with with a lot of you know understanding a lot of you know Libraries that we love and and use on a day-to-day basis use this standard or you know Even suggest a standard or use the standard internally for you know, the artifacts themselves So then this raises the question of like well, you know What should what should be done for this specific context should we introduce some like super smart type? Systems that allows us to scan pickles for Vulnerabilities, well, you know the best practice tends to be similar to general Containers or loading any type of binary code from any resource is the concept of trust or discard, right? You know similar to docker images, you know, you need to make sure that the Location where you're running where you're fetching those containers that have arbitrary code or not arbitrary But like the potentially arbitrary code, you know You can trust those those locations and of course you also can trust the pipelines that generate those artifacts, right? So if that CICD pipeline is compromised, then you may be compromised, you know down the stream in different areas So we're gonna revisit some of this this points The second part that we're gonna now talk about is access to the model on unrestricted access to the model So if we deploy a model and we don't restrict the access that means that malicious players would be able to access and Consume that model to their own liking and one of the interesting areas that we have seen is the concept of Adversarial attacks, so this is basically being able to generate Adversarial examples that can trick the machine learning model whilst also tricking a human So we actually have an interesting example that you can try out with one of an open source libraries called alibi detect So this shows you not only how to create adversarial examples, but also how to create an adversarial detector, right? So that you can put some sort of like, you know Advanced monitoring component to identify if an example itself and input is is adversarial So and here you can actually see how some of the adversarial examples can be created And you can also see how those can be used to trick the machine learning model themselves So that's that's one interesting thing that I would want to point you out and for you to try it out The second one is that if you have access on restricted access to the model artifact itself It is also there's also interesting literature that shows how it's possible to extract the training data from you know Things like large language models themselves So this is another consideration that may you know raise potential privacy data privacy and data leakage Challenges when it comes to under stricted access to the model So that's one one one key consideration moving to the next part is code, right? We tend to forget that as practitioners Maybe we are writing Jupiter notebooks and we may think well That's actually you know not the same as like writing a web application, but once it gets to us a certain you know, I guess Use or scale that actually becomes code or analogous to code itself And it has the same potential challenges that you would find with normal software that you have So code code scan tools are very useful for those type of situations And especially because we have seen in some contexts some of the vulnerabilities that have been highlighted I don't know if people here remember about the vulnerability in the pie in the in the pie YAML package So that actually affected a lot of libraries and you know This is an example that you know practitioners in the data science space may not be aware of of that So, you know from code scans you can use a lot of really interesting tools You know, we actually show how you can use tools like bandit So bandit allows you to actually search across your Python project and identify any vulnerabilities low medium large etc For that so now on the on the next, you know highlights is making sure that you know This also is in compass for Jupiter notebooks Now dependencies are important right now There's a lot of discussion about the supply chain vulnerabilities So if you pull a dependency in Python and you pin your dependencies that doesn't mean that your second third fourth level dependencies are going to be pinned and We actually these are actually vulnerable to supply chain attacks We have actually seen a lot of cases recently where you know pipe-by-packages have been compromised and you know maintainers may actually End up in a situation where that is like a fifth seventh level dependency and that may be highlighted So it's important to be able to address this and we actually have worked with some projects like ML flow To ensure that you know some of these things that were done dynamically of pulling dependencies now It's actually done at once and one of the things that we often try to emphasize is for the pennancy scans There's a lot of tools that you can leverage right so one of them is safety So that's like a Python package that you can leverage and also other packages like pip depth tree to be able to identify What are all of the dependencies that you have now there is tools also like poetry that allow you to create lock files Which basically you know means that you can have an understanding of you know all of your second third fourth level dependencies So you have like fully reproducible environments. So this is an example of a poetry lock file So that there are a lot of resources and that is the emphasis that right now is practitioners You can leverage and you know recently this was a conversation I had with one of my teammates, you know Highlighting some of the challenges where everything was working on a Friday and they go broken on a Monday when nothing changed Finally the last part is on image scans So this is one important thing especially for us that we have to carry out You know the deployment of this you know servers so that people can deploy their models We actually have to run the automation of these scans With every release and we use tools like trivy that you can leverage yourself So this is like you know a free tool that you can you know Leverage in your own CI CD pipeline and you can search for CVEs across your image So for example, you know this morning we identify a CVE and we had to just rerun the image build To address it with the latest package. So that is that is a set of examples You know, this is of course a python conference So I'm not going to bore you with kubernetes related shenanigans But it is equally important to have some of those you know components in your infrastructure Right and that means things like encryption on data and transit in a data address as well as authentication authorization Etc etc Now if we actually look back at some of those components that we that we defined initially with the OWASP top 10 We can now define what we can call the omlsp right the open machine learning Standards project and we have some of those one-to-one mappings between things like broken access control to access to model endpoints cryptographic failures to Access to model artifacts etc etc and we've actually kicked off An initiative within the linux foundation to define some of these things through a machine learning a machine learning security committee So again, this is a more of a call to action to emphasize that we're currently all looking to build those best practices Now I didn't want to stand here and just like raise problems I also wanted to you know emphasize that we are also trying to find some solutions and best practices. So we created a um, I guess Analogous repo to the flood machine learning security. So the fml repo, which is called sml so safe machine learning security Which is basically a base template that you can use to generate a starter I guess machine learning engineering package and what that means is that you can just basically like start it Like such you basically, you know, just run with cookie cutter. It's like a template generation project. Let's call it like example project Then you know, you put things like your your repo username The name of the project the number of the name of the classes the description The license that you want to use and that basically will generate you that base project and the base project itself What that would have is it would end up with you know, a base server Together with the dependency package management The scans for security that you can run as part of your ci as well as you know Your runtime environment that you can leverage to package your machine learning model using this framework called ml server So the ones that we were using as part of this as part of this talk So just to wrap up on some of the things that we've covered If we take a step back, we've talked about technology But it's important to make sure that human and process is heavily involved when we look at production machine learning systems The architectural blueprints always have similar steps of experimentation Taking training data to train models and create artifacts the sort of automation or manual deployment of models Whether it's ci cd or through, you know pipelines to create, you know, running either batch or real-time models advanced monitoring components for the for the, you know, things that we mentioned like adversarial Robustness adversarial detection drip detection outlier detection and then making sure that there's full continuity with some of the data That is part of that the processes are important as each of these areas as you have a data scientist that is creating those models You have a machine learning engineer that is creating those use case automation pipelines And then making sure that there are also those deployment pipelines so that you can have the inference across your your models As well as the DevOps personas that would be involved and security is not just about tools It's about the processes that have to be integrated But also that are proportionate to the maturity of your organization and the risk of the use case Right if you are just building a small prototype with perhaps just a hand, you know a small group of You know practitioners the actual process can be enforced just within this unit, but as it grows standardization, you know Centralization control of the production environment starts becoming more important as well as role-based access control for the for the data that flows So for anyone that is interested on deeper dives, I know that a lot of the deployment stuff was very hand wavy There are other talks to deep dive into things like CICD for production machine learning For machine learning monitoring for machine learning acceleration, etc. So please Do have a look and just a reminder you can find all of the resources in the git repo And the slides at the link at the bit link on the top right So with that, thank you everybody and I hope you enjoy the talks. I'll pause for questions now Awesome. So if there's all right, I guess people come into the mic. Yes. Thank you for the talk Yeah, you mentioned the trust is important to have A trusted pipeline that generates the the model Binary, let's say, but actually that's nowadays inverted that it's important to have zero trust So, uh, can you maybe talk a little bit closer to the mic? Well, you say it's about trust, right? Trusted pipeline that generates a trusted binary in that sense But I think nowadays security is actually more about zero trust Because if you trust the system, then you're open to attack So it's more about verifying your security instead of assuming it Yeah, I mean that sounds more like a like a thought or a comment. Yeah, I mean, I agree. I think I think when it comes to Yeah, security there are probably like two parts to this one is just having a baseline of best practice And the other one is as you mentioned verification, you know over testing itself But I think a lot of these concepts probably focus more on the on the former, which is just What are the basic set of Minimum best practices that machine learning projects should be conscious of You know as it as you're dealing with Scale, especially with more critical use cases. So so yeah, I mean, I would agree Yeah, and I mean, you don't have any suggestion. What would mean the model binary verification then or because I mean you can It's more the subject that you touched but not really any Guidelines, you know, yeah No, I think I think it goes back to that point that was made On emphasizing trust or discard I mean, I think it is possible to go into a sort of like, you know exploration Of advanced methods to be able to verify the binaries themselves But, you know, it's the same sort of paradigm that you have with things like docker containers You're still going to be pulling things that are analogous to You know components that have Code that would be executed So you have kind of a limited amount of how much verification you're going to do throughout each of these areas So, yeah, I think the main emphasis at this point is more that trust or discard mechanism There are some interesting projects that explore actually binary like pickle scanning Specifically in the machine learning space, but again, those are still in the very early exploratory phases Thanks for the talk So we talked about dependencies and one thing particularly that happens with machine learning right now is that everyone builds their own model But now with the large language models, we're moving to have another models as dependencies and Fine-tuning other models into a new one and then over and over again What happens someone to Someone from manipulate instead of fine-tuning manipulating those parameters Uh to trigger not really perhaps not remote the execution or whatever but unintended side effects on the execution of those parameters of that prediction The challenge here is that when you do the penises scanning or you go through The penises upgrade the penises to have a track of what was changed on those dependencies And that doesn't exist anymore on is not as human readable Anymore on the machine learning side because you only see numbers changing you don't see it's a lot more similar to I don't know assembly code being changed or anything How do you see this in a few years from now when Machine learning becomes as much as building software where you depend on a lot of dependencies of machine other machine learning models instead of just like building your own all the all the time Yeah, well, I mean I would say that that's the case now. Um, you know, we we see that already causing Introducing risks in production So the tool that I was showing you a seldom course that allows you to deploy inference graphs with multiple stages So you can have like, you know preprocessors post processors each of them with like different versions different frameworks different libraries Um, I think that that is why right now there is such an important such a critical emphasis On start on thinking about the best practices around security So although I agree that, um, you know, it does become quite almost impossible to to to go into the You know code specific change Across every sort of like, uh, you know dependency modification throughout your supply chain That's why it is important to leverage some of the best practices that have been created in the software side Which is leveraging things like CVE scanning and cv databases So that actually keeps track of some of the common vulnerabilities Exploits as well as remediations And that is something that will only grow in the machine learning space making sure that Perhaps even some of those cvs are tailored specific to to machine learning type use cases Perhaps even for things like adversarial robustness or, you know, perhaps even like privacy preserving areas That that may result in in risk So it it's more about bringing what is already Being used in practice in production as opposed to reinventing the wheel So that's what I would suggest and that's how we are approaching it, right? That's what we we were taking the OWASP and trying to see well, what do we take? What do we leave what what what needs to be rethought? Thanks also from my side I have a question about the possibility to Reverse engineer machine learning models because I often hear people being afraid, especially when dealing with sensitive data That they are afraid of the possibility Of reverse engineering the machine learning model or even the data itself that was used with and I was wondering if you have any insights on that Yeah, absolutely. So so I think that was one of the the research papers that Yeah, was linked here that that you may be quite interested about So so this one actually explores that exactly is is reverse engineering Um, I guess training data from from large language models, uh, which you know, in in some cases may include Potentially personal identifiable data or or you know critical data So this is certainly a risk and I think that is why it's important to emphasize that when it comes to security Is not just about you know verification of an artifact at a particular point in time It's making sure that you know, you're leveraging best practices throughout your entire operational Touchpoints with the stakeholders that are involved, right that your ci pipeline is is is robust is It has a relevant role-based access control. It has the relevant practitioners, you know Involving the development and extension of that And then the same way for the management of artifacts making sure that those Are, uh, you know restricted in terms of access Similar to how you have restriction of access for for data, especially critical data So it's just introducing those best practices, but To the broader set of artifacts that are involved in the machine learning life cycles All right All right guys that was that was an amazing session and a very insightful q&a, but unfortunately we're On time so like we've just run out of time. So yes, uh, you guys can move into the coffee area and Like ask more questions over coffee So thank you