 Awesome. Hi everybody. Hello. Good afternoon. And if I may, bonjour. So welcome to the last day of KubeCon and to our session where we will talk about how I met your software. And disclaimer, we are slightly inspired by how I met your mother and promise we won't take it too far. Of course, I will thank you all first. This is such a lovely turnout for coming here after lunch. That too on the last day. We'll try to make it worth it and we are excited to have the next 30 minutes of a lot of fun and interaction. Thanks, guys. Wait, wait, wait, wait, wait. Why is the software spelled as software like that? Well, AI, AI, AI. I think KubeCon has talked a lot about AI. So don't mind. AI is still not taking your jobs. You are here to fix the sparrings. So we have to move ahead. So I'm sorry if I'm going to call you all kids. You're all certainly not kids, but kids, we are in the spring of 2024 and here we are going to embark on our journey of securing or securely consuming your software. And before we embark on our journey, let's briefly introduce about ourselves. Hi again. I am Anushka Mital. I work with Nirmata and I predominantly contribute to Kibirno. Hi everyone. My name is Mithunjay and I'm a software engineer at Chain Guard and I've been contributing to projects like Wolfie and Kibirno. And other than that, you can also find me making some cappuccinos. Hi everyone. I am Subash Mata. I work with SIWO as a research and development engineer. And I occasionally also contribute to Kubernetes. I'm part of the CI release and release Kubernetes release team. So I work with the CI signal team. Thank you. Let's move ahead. Okay. So by a show of hands, how many of you think there's some difference there like jellyfish? Jellyfish. By a show of hands, of course. Oh, okay. So it's probably, thank you. Do you think you can point out to difference? Wow. Clothes actually. You are close. Not really. Not close, but not. Anyone else? Any guesses? Okay. Okay. So I think now it's turned to reveal what's the difference here. So probably if you were in 2019 or 2020, and probably if you were installing by just copy pasting a command, you might have installed a malware that has till your SSH and GPG keys if you have installed the second one. Can you move to the next slide, please and show what is the difference there? It's I. So, so these kind of typos, squatting attacks have been quite common when you have been, if you consume projects, if you consume software directly to put this like Pipeye and Pipeye has fixed it more than six times. I guess this was a six time when this was fixed, but it caused a lot of repercussions because jellyfish was also used with date util, with Pipeye date util, which is again a clone of original date util. So these are both the packages actually worked quite the same because they were just copy paste, but the second one was actually stealing your GPG and SSH keys. Now, since we have set the stage, you guys love stories, right? And since our theme is also based on sitcom, I'll probably share with you one story. I love Christmas, okay, and I love Christmas surprises. So let me walk you through a story, right? What happened on December 25th of 2022 when everyone was busy enjoying their Christmas parties, something traditional or you can say a legendary incident happened, right? So a package, if you know, PyTorch, PyTorch also has a dependency called Torch Triton. What happened was the Torch Triton, a malicious package disguised as Torch Triton, was included in the Pipeye. Anyone who installed Torch Triton via pip, just pip install that Torch Triton, it actually installed the malicious package. How did it happen? So Pipeye has this open precedence when anybody can upload a package with a similar name. And Pipeye is designed predominantly to use the first version or the latest version by default. So the latest package, the malicious package that got uploaded to the repository of Pipeye, it got installed to users computers by default. I mean, it looks the same, it spells the same, but it is a malicious binary. It could include, it includes your, it could read your passwords, it could read your SSH keys, it could also like read your Git conflicts. And it also had the ability to package all of that stuff into a file and upload it to an unsecured website. That is so, so scary, right? I mean, that's not a bad Christmas surprise, but it was for the hacker, of course. But it was a nightmare for other developers who have, they have broken. Yeah, yeah, yeah. They broke their own security, they've breached it and they are doomed now. What should they do now? They are surprised, they were surprised, how do, how do they handle it? How did they handle it? Yeah, so, you know, that was really a horror story. Glad I was just graduating back then. So, all right, you know, let me put some numbers as well. This malicious software was out there for five days available to be downloaded and used. I think by the end, more than 2,300 downloads were done of this set of malicious software and given what it could do, that could have been crazy. So how did PyTorch sort of, you know, help with resolving it? So they of course renamed the package internally. The second thing, they deleted any dependent packages, any dependent on Torchetron packages. They also reached out to the PyPy security team to get proper ownership of the Torchetron package and they, you know, deleted the malicious one. Finally, they did provide some support to find out if you were affected or not and what you could do. But really, this is something that happened two years ago in 2022 and this is just one of the many attacks that we've seen. This we've used to really set some context and understand and remind you how important security can be, getting secure software, keeping your workloads secure and how important it is to really have a robust and secure supply chain. So that was a little bit serious. Now, we have a friend, our friend is Ted and this is not Ted. Ted is somewhere, somewhere. So Ted is a determined guy who wanted to find the one, the one right software secure for him that made him feel safe. So let's see his journey. Okay, before, before going to see his journey, I would like to introduce you like what my idea is of consuming software, right? There are multiple options. What do we choose? Do we go via distributions like Wolfie or Alpine or Debian? Or do we do via projects like PyPy and go like we discussed, right? Let's walk through. So if I choose distribution first, we get the pre-Pakistan installation, that's great. We get to hand handle it and we get great user experience and we can tailor it for our use case, right? But then that compromises the user system. It will give right direct access to your system. But if we consume it via customized project, what will happen then? I mean, it's great. You can tailor make it for your own use case, but still you open doors for compromising your own system. Okay, let's see, maybe via configured project that's already there. People are maintaining it. We can trust people, right? We can trust maintainers. They are great. They're doing great jobs. What will happen then? It goes into the supply chain, software supply chain, right? And just as we discussed before, this PyTorch incident, anyone who wants with not good as intentions as we have sharing the insights with you here can push the package to that software supply chain, right? And it will intentionally break it and make it malicious. It directly goes to your user system and it compromises it. I mean, we have so many options, but none of these seem to work right. What should we do now? What should we do? What should Ted do? Thank you so much, Shubhash Mehta and Anushka for setting the stage. But now, our friend Ted has been traumatized. And this trauma is leading him to a lot of retrospect and a lot of thinking. Like what should he do next? Yeah, I mean, should he go for secure software? Would that probably be latest software? Or probably he should look into more distribution or packages that are available. I don't know. Okay, wait, wait, wait. Let me take a lead from here and tell you what this dilemma can lead to and how we can actually solve it. Enough of talking about the problem. Now let's talk about what are the solutions and what are the pathways. So developers often face a dilemma that, hey, whether I should consume a project via distribution or whether I should consume directly from PyPy or NPM. Especially with node packages that what we are going to discuss next. So before I move ahead, let me ask a couple of questions to my friends. So Shubhash Mehta, have you ever consumed a Python project? And if you have consumed, what has been experienced whether you have used PyPy or Anaconda? I mean, well, I can share my insights, the pros and cons for Anaconda, consuming it by Anaconda or PyPy. And you guys are smart. You can decide then. Okay, so Anaconda offers a set of predominant or predefined packages that are there. And the curation is maintained by the Anaconda team. And it has reduced the risk of installing malicious or fully maintained packages, right? The another advantage of Anaconda is Anaconda maintains isolation package environment where you reduce the risk of dependency. Like mismatch dependency conflicts. And you get really like the latest software, not unintended software. Additional security in Anaconda includes something like packet signing, checks some verifications, and it ensures the package integrity, right? On the other hand, PyPy has this open feature and it holds a vast number of Python packages. So in terms of variety, you can look into PyPy, but in terms of very, very secure software that you don't want to compromise on. Anaconda would be a better choice. So PyPy's open nature would allow anyone to upload packages as we discussed before, right? But however, PyPy also has mechanisms input to reduce the risk that we have at hand. That is why we're here to discuss all of this stuff, yeah. Thank you so much, Shubhash Mithra. My next question goes to Anushka. So now that we have answered over Python packages, the more interesting one comes with, how do you deal with Node? How do you install Node-based things on your system? I would probably use NPM, the Node Package Manager, to get different packages and maintain its dependencies. And I think that's the only one I've used before. Yeah, I mean, that is one of the things that we have often faced is that while Python still has Anaconda, when it comes to Node-based applications, we usually prefer NPM. And again, that comes with the problem of pushing anything that you want. And again, if it's not audited and it's not checked properly, you might be conserving malware. There are distributions that have come up which are trying to solve even for this. And one of them is like Wolfi, where you can even install NPM packages. And in fact, what I'm going to talk about next is a lot of comparisons between different distributions that we have. And each distribution has its own benefits, advantages, and disadvantages. But one of the tussles that we often see is G-Lib C versus muscle. I mean, not this muscle, of course, but the standard library of C. So let's talk about whenever we are building images or whenever we are building software, we often end up using Alpine as base layer or we often use Debian or sometimes we use other options like Wolfi, which are new. So Alpine is often one of the most commonly used because it offers minimal container configuration. It's more secure, but it comes at the cost of using MUSL. And what happens when you use MUSL is that DNS is always the issue. So it probably is one of the most common problems of using MUSL is that it does not, by design, allow DNS and prefers it over and it's not preferred there. And that has led to issues like especially if you are building an application, it runs fine on your local machine. But it is very much possible that when you run that inside Kubernetes clusters, you might face and hit that error again. Now, what happens when you deal with such issues? It becomes problematic. While Alpine is great, it is very much important that in such cases where you are stuck, you have an alternative for yourself to actually consider. And Wolfi is actually an undistro. I mean, undistro calling something undistro is quite contradictory or quite an ironical nature to say that. But it is because Wolfi does not have a Linux kernel itself. It operates and it uses your own container. It uses your own host and it's basically a containerized Linux environment that ultimately makes you run your, basically install your packages that you need. And it's very much Alpine based. It's not something that you might have to do differently. It's all APK that you might have to do. But that is a tussle that we often see and we wanted to discuss that when we are talking about different distributions. Because as we discussed through Subash Mith and Anushka, we often complain, what should we do? I think ultimately developers should build and distributions would distribute in simple words. And that is where ultimately Unix based distribution is simply a tar.z. And if we make it as simple as for the end users to consume via distributions, what it offers better is more reliability, more security, because dedicated volunteering and maintenance are there for different distributions, whether it is Debian, whether it is Alpine, whether it is Fedora, whether it is Wolfie. And one of the problems that developers often face is that, hey, they are not going with the latest software. They are still stuck on something else. They are not following the speed. And that is what limits them to sometimes consume the process directly. And that is what sometimes makes a problem for them. But that is also being solved with things like Debian recently and in fact Wolfie. Wolfie has a specific GitHub action bot for that, which always checks that whenever the latest update comes, how we make sure that it is always up to date, the software that is passed and is latest up to date. Coming next, I think our friend Ted is still looking for Redflex, right? He's a determined guy. He needs to be sure. So yeah, I think it's time that probably we show you some example of pulling a public upstream image from a project directly, like SQL pad. And how if we try to scan a CVS with some distribution and what difference does it make? So probably let me begin by doing that. So let's start with SQL pad is a web-based editor and for SQL pad. And now I'm going to do a simple gripe check. The gripe is a scanner for checking the CVS and I'm going to pull its public image and going to compare with a Wolfie based package of the same. Please bear with us. So if we just look at it, there are around 90 vulnerabilities and some of them are coming from NPM, some of them are Debian based. But yeah, one of them is critical that's coming from Debian. And that is where the part that we were discussing comes into picture. Now let's compare it with something when we talk about a distribution. So this is a Wolfie distribution that I have been learning locally. And if I do APK add, it's already installed for me because the internet was causing a problem so I have already installed it, but let me try and do a gripe scan on this package. No vulnerabilities found. I think Ted can now be a little bit convinced. Let's move to the slide of what he might be going to choose. Yeah, I mean, Ted was realizing his patterns. He saw that he could probably, you know, make a final decision. So Ted was happy. He found the one. Awesome. So that was pretty much like the second part of our session. Let's move on to the last part. And what's more, you know, you've been around for three days. You've probably seen multiple solutions out there, multiple things that people are doing for security in general overall. So let me just talk about a few. I might of course miss some, sorry. But all right. So the first thing that Mrytun judges demoed was, you know, embracing de-strollist distributions. So this will sort of, you know, just like Wolfie, it helps you get container images that are more in tune with the requirements of modern supply chain security. Just being, you know, using signing and you know, using cosine or notary or projects to sign your images. So I'm a developer. I've built something. I've made an image. I want to sign it because at that time I knew that this was safe and secure. So the next person using this image, verifying this image can be sure that this is safe. Right. So that's using signing software. The next would be to use a policy engine. So you have your workloads running. You have everything in place. Let's have policy engines. Let's have admission controllers that make sure, you know, no request is a bad request. There's no bad configuration. Examples of policy engines, of course include Kivirno, Opa gatekeeper. What these will essentially help you do is validate all your requests coming in your workload and make sure they're compliant. Of course, just FYI, because I'm involved there, Kivirno, traditionally known to be Kubernetes native can actually now be used across cloud to scan for misconfigurations. All right. So you can, of course, you know, enhance container security using tools like Falco. The Falco is a threat detection engine. You shouldn't probably, you don't need to look into zero trust in networking when nobody is trusted by default. Of course, embracing DevSecOps practices, and this should be in all stages. You in the commit stage, in the build, deploy, and run. You need to have practices, DevSecOps practices in all these stages. And yeah, of course, there's, you know, you want to utilize service mesh. I don't know why that looks. It's a mesh. Yeah, so this mesh technology is like, it's to link her to have, you know, proper service to service communication security there. And, you know, this line really concludes everything for me. You need to need to have robust strength and secure supply chains. And you need to be able to depend on them to make sure nothing bad's happening. Like we have discussed this on general terms. These are like also depend on what things you want. What are your priorities? So those can be boiled down to those aspects as well. How you're going to consume that on a lower level or a higher level. It also depends on what priorities you have. What advantages you want to look into. That all depends on the user use case, right? We are discussing it on a general basis. Yeah, definitely. Moving over. I think we ran it quick. We are short of slides, but we did share it. No, I mean, it was a quick journey. Thank you. And that's kids how I met your software. That's us. Thank you, folks. And we are welcome to connect with us. So if you wish to connect with us, these are QR codes. So we would love to be connected with you and discuss more about any feedback related to talk, anything that we can improve. And just to talk, of course, yeah, this is probably the coolest slide. It has QR codes. OK, all right. I think we have a decent amount of time to take some questions if there are any. And I do realize if the last day just after lunch, you might just want to, you know, get some free time, get a nap or something. But yeah, we are round if you want to chat. And discuss more about this topic. Yeah, we'd love to. So I guess that's all right. Thank you, folks. Thank you for coming.