 אני זיו, אני סייניור סיינטיסטי סיינטיסטי בי-בי-אם-ריסטרצ'יינסרל והסתוק הזה היה מלמד בגמרי שמוליג פוימוביץ' שאולי לא יכול להיות כאן עכשיו הסתוק היתר הוא פוטינג-הקרס בריג'ינגיו קלאסטרל עם אוטומטיקה ורנטין, אז אתם יכולים לראות, זו סתוק על סייארטי ואני אמרת שסייארטי שצריך להסתכל על סייארטי הוא פוטינג-הקרס בי-הקרס, שאולי נפלאה את זה אז כאן יש לי פוטינג-הקרס באמת, יש לי שם כזה בבקשה וזה נשטי, הם מאוד מפגל וכך כך זה חייב אז לא נתקדת את זה אבל אני עשה את זה, ועכשיו, מה שצריך הוא שני פרקים של קלאסטרל ‫הוא לא נעבור היגרס קטור. ‫לא, זה נכון. ‫אתה לא נכון enough, ‫זאת אומרת לי שזה מאוד נכון. ‫בכיוון, כי בין היגרס קטור, ‫את האפליקציה היא עושה ‫בכל כל מיני עבורים. ‫באללה, זאת אומרת, ‫העבור של עבורים ודתליקים. ‫אם אתה לא נכון מה נעבור ‫את הולך, ‫בכל כל מה נעבור את הולך. ‫הוא מאוד איוצרBLE, ‫תם ככה מלכים ‫שכיחים על אני בין שתכף ‫לחסב כרוכים במתערבי. ‫אז יקרם, כשתכף אם ‫אני נקרן שבו כתובעתMYPS, ‫זה כרוכתי, אבל יש נפטר, תלך נכון גם הנביאים ואתם יודעים שתרוצים הן אישרות שיעריים בלבים נחמד בן אחד שזה גם מגיע תן שלא עביץ בלבים עמה וא letzса Nick איתרים folks who attended the security con may heard the very frightening number of 742% of trice in supply chain attacks in the last 3 years So this is really scary, so basically it's not a question of if it's a question of when your application will be hacked and you should be prepared and at least protect your egress, so data doesn't get out so easily. So this is the customer's slide with some recent attacks that maybe could be prevented with tighter egress control. Oh, and there's also compliance. So these days you have standards and NIST SC7 tells you that you should deny network communications traffic by default and allow network communications traffic by exception at managed interfaces. And only those system connections which are essential and approve it are allowed and you may say, okay, but I closed the egress side, but this doesn't mean that a microservice of yours cannot talk to the outside internet and so you must control also its egress. So why is it that egress is so exposed? So first we have lack of awareness. Hopefully now you are aware that this is something that you should think about and do something. Then people tend to trust cloud vendors to have the protection, to put the protection for them. Well, this is usually a bad idea. Cloud vendors can't really close all your egress or otherwise things are not going to work and they cannot prepare for all use cases. Now, the third reason might be that you say to yourself, I didn't do much. I just took WordPress and MySQL and Prometheus and Gafana and put them all together. I should be okay. They know what they are doing, but they know what they are doing, but this is not enough and I will have more details about it soon. Now, putting egress control is considered hard to configure and to maintain and this is actually true. This is not trivial. And also sometimes you do have egress control and you do limit your connections, but the configuration is not really what you meant. So the configuration is wrong and maybe you are not even aware that this is the case. Finally, in a development team it is really unclear whose task is it to put the protections in place. Is it the development people? Is it the developers? Well, they usually don't care much about security. They just want their code to work in place. Is it the developers people? Well, they are not really aware of which connections should be in your application and so they may not be the right people, maybe the security people. Well, they don't really, they know the rules. They are not always sure how these rules should be applied. So my claim is that for the last three bullets, actually automation can help us here. And basically this is the agenda for the rest of my talk. I will show you why trusting third party components may not be such a good idea and I'll show you how automation can help. Just one word before I continue into that. So where is the best place actually to control egress? You may have several choices. Maybe the cloud provider firewall or the VPC network ACLs or security groups. Maybe the cluster, maybe you have an egress controller for your cluster. Well, I think that it is best to put egress as close as possible to where your data lives. All the other places, well, it may be too late to stop data leaks at higher levels because they are not specific enough for the kind of communications that are really allowed. So in Kubernetes-based application this actually means that you want to control pod traffic. And the way to do it in Kubernetes, natively, is network policies. There are also CNI-specific network policies, like Celium network policies or Calico network policies. But these are CNI-specific and if one day you want to migrate from one CNI to the other, then you have to migrate. There are also service mesh-specific network policies, like Istio authorization policies. However, these are layer 7 only and consider less secured. So now that we know about network policies, let's see. So we made a small survey of some common open source packages. So we were looking at 15 of the most popular Helm packages from Artifact Hub and Bitnami, including databases, registries, event systems, all the packages that you like and imagine here. I will not get into names. The good news is that most come with some ingress control and this is good, although typically you are required to put some non-default flag you have to put it on. But that's the good news. The bad news is that only 4 of these packages out of the 15 that we looked at, only 4 included some sort of ingress control. And remind you, this includes databases. So if your database has no ingress control, it might be that your pods can send data directly to the Internet outside, to hackers. It's not a good idea. So never trust other security or as we call it, zero trust. Not to be too pessimistic, here is one example where things are done right. So you can see here that the two stateful sets in that case talk to each other in a restricted way and are connected to the outside world only through UDP 53. So this is the only ingress that is allowed. This is DNS. This is considered fine. So you may ask, well, okay, so this is a very nice graph. How did I get this graph? How do I know if my package is doing things all right or not? So this is where I said automation can help. And there are tools to help you with this observability. So here is an incomplete list of tools. These are the tools that I know about. There might be some more. And there are tools that can look at your live cluster and let you know the connectivity posture. So some of these tools can only look at the actual traffic that flows in your cluster. So they may not be aware of hypothetical traffic that can go out of your cluster, but this is definitely better than nothing. At least know where you stand. Other tools can actually analyze network policies and tell you what is the allowed traffic and what is denied. So we have here tools like Red Hat ACS or Starkrocks. We have Sysdig Secure, Selium Hubble, Calico Enterprise, and NPGuard, which is an open source project for which I am a maintainer. Now NPGuard can also look at your repo and tell you what is the expected connectivity posture for your application. Given the YAML manifests that live in your repo. So this actually supports shift-left methodologies. So this is about knowing where you stand, but how do I get to the place where my connectivity is done right? In here automation can help once again. Automation can help you synthesize network policies. And there are two kinds of tools here. One kind is tools that look at the actual traffic in your live cluster and use it as a baseline for synthesizing network policies. So Red Hat ACS and Sysdig Secure can do exactly that. The other type of tools is tools that are analyzing your YAML manifests and understand what is the required connectivity from these YAML manifests. And when they know the required connectivity, they can synthesize network policies that will only allow this required connectivity and nothing more. So NPGuard is once again one of these tools. And the technology used by NPGuard is now going into Starcox, which is the upstream for Red Hat ACS and will be included in Red Hat ACS as well soon. So NPGuard philosophy, as you can see, is doing things on the left. It's configuring the connectivity on the left. And the vision is made of two phases. So there is the analysis and synthesis phase in which network policies are being synthesized and this is proposed as a network configuration for the user to review. Now NPGuard will not just provide YAMLs to the user while these are network policies take a look. We want more than that, so we can provide the connectivity maps that we've seen before. We can provide connectivity diffs. We can tell you what is changing in terms of connectivity, not a syntactic diff of YAMLs, but rather telling you which connections will no longer be available after you merge this pull request and which connections will be added, which connections will be modified. And there is also the issue of organizational policies. So you will see some material that some protocols or some ports should be blocked or maybe there is a denialist for outside IPs. And these can be specified as rules and they can be used both in the synthesis part so we don't synthesize anything that is not compliant and they will be rechecked at the review process even if the user chooses to add more connections because maybe the synthesis missed something, then any change to the network policies will be rechecked against the organizational rules. And only when the network configuration is compliant only then can it be deployed to the live cluster. So let's go ahead and see a demo of how this can work. So this use case is a Greenfield use case, so we have no network policies at all and I decided now I want to have some protection. So I will initiate this analysis and synthesis step for the workflow. This will generate network policies, open a pull request with the generated network policies and other workflows will annotate this pull request with the connectivity map, the connectivity div, and we'll check that the connectivity is compliant with some high level rules. So let's see a demo. I should... One second. Okay. Is it on? No? Okay. So this is the online boutique example made by Google, a very popular example. So this is basically an online shop for vintage products and here is the architecture for this application. It is made of 11 also microservices that talk to each other in a non-trivial way. However, this application comes with no network policies, so no protection at all and for example the front end here, if a hacker can get from the internet into the front end here, then the front end can directly access the payment service without going through the checkout service first. Moreover, the payment service can actually talk to the outside internet. There is no protection at all here, so data can leak from the payment service, all the credit card numbers and so on. So we want to apply network policies. So we have here integration, we integrated it with a grid of actions in that case. There are also integrations with tecton tasks and it can also be consumed, the components can also be consumed as docker containers. I will go to the synthesis workflow here and maybe we first take a look at the workflow and see what we have here. So basically what this workflow is doing, it checks out the code, the application code and the manifest and called the NPGuard Netpol synthesis GitHub Action and we then use the artifact or the synthesized network policies that were produced by NPGuard and commit them to a branch and open a pull request. So this is how it works. So let's run. Let's hope this is going to work. In the last five days, I've seen a remarkable number of failing live demos, but I like living on the edge. It's me and the centipedes. Okay, so yeah, it runs, it should take around 30 seconds and once again, this checks out the application manifest, it analyzes the manifest to understand what is the required connectivity and synthesizes network policies to allow only this connectivity. Okay, so it is done and it has opened the pull request for me, which is very nice and let's see what it has done. So it actually added one file with network policies. So we can see here one network policy per microservice. So for the checkout service, we have this network policy saying that it can connect to the email service, the payment service, the product retailer and so on and so on and so on. And it can get ingress from the front end and the email service here can get connections from the checkout service and so on. But no one likes to look at YAMLs, right? YAMLs are not easy to understand. Hopefully, I should have here if I refresh, I guess. Yeah, cool. So I got a few pull request comments here that let me know what is my current status. So here's all the allowed connections. I have here a pull request comment with all the allowed connections in my application house. I can see that the checkout service, the front end and the recommendation service can connect to the product catalog service by using TCP3550 and the checkout service and the front end can talk to the shipping service using this port and we can also see that it is only the front end that is exposed to the outside world which is cool. This is really what we want and we can also get a connectivity map like that which is much easier to understand but tables are also useful, especially informally. Okay, so that's cool. What else have you got? So there is also this diff here. So another pull request comment that is telling me the difference between the previous state of my application and what's in this pull request. Now since the previous state was that everything was allowed, all connections were allowed in the cluster. So many many removed connections when I applied these network policies. So you can see here that many connections are no longer available both in cluster and also from outside of the cluster to the cluster. So all these connections are no longer available which is good. This is what I wanted. I wanted to remove many redundant connections and leave only the connections that are really needed. I have here I set a few high-level rules. Sorry, it's not that. It's here. Another pull request comment. This is checking that my connectivity is compliant with several high-level rules. So my CISO, for example, told me that no FTP is allowed in the cluster, no Telnet, no SMTP, no non-secure protocols and all these rules actually pass which is nice. But I have here a failing rule. So there is another rule that every microservice that would like to access the payment service should have a specific label. And this rule is failing here and we can see the details here. The checkout service is trying to access the payment service using this protocol and port but it doesn't have the required label. So you may ask why did synthesis allow such failure? Well, the reason is that I didn't provide this rule to the synthesis to the synthesis workflow because I wanted to show you a failing rule and this rule actually blocks this pull request from being merged which is what I want. I don't want any network configs that are not compliant with the rules. So let's get back. Okay. So once again, this was like a Greenfield use case. I started and I had an application with no network policies and I started the analysis and synthesis workflow which generated network policies, put them into a pull request and other workflows put pull request comments to show me the connectivity map, the connectivity diff and the organizational policy so to summarize, securing egress is not enough. You should also always secure egress and the best place to do it is at the microservice level. It is important that you will always know your connectivity posture and there are tools to get it. So I mentioned the list of tools there might be others but do know where you stand Also, there are tools to help you with synthesizing network policies, suggesting network policies. These tools may not be perfect yet. You have to look at what they produce and make sure that they don't miss any required connections but they will take you to a good place to start with and it is possible to integrate both synthesis and analysis into your CICD pipelines so you can have a shift left methodology that will make sure that your connectivity is secure. So that's basically it. One more advice. Do wear high top shoes when centipedes are around. That's important. And if there are any questions I'll be happy to answer. Wishings. We're at me if you have questions. Thank you very much then.