 Good morning everyone welcome to cloud native security con and welcome to our talk on what is zero trust Or what is the zero trust tunnel? We're gonna dive into ambient mesh and how We're able to provide a simpler approach to security operations with ambient mesh Come on work Go ahead. There we go. Hey. Good morning everyone. My name is Jim Barton. I'm a field engineer here in North America with solo And yeah happy to be here this morning Awesome, I'm Marina Wiget. I am a platform advocate at solo There's a lot of things you could do you could follow me if you want But one thing I will say is if you're kind of new to the Istio journey or the service mesh journey I run something called 70 days of service mesh and I'm doing this on github. You can follow along I basically will dive into various topics of the world of service mesh and an Istio and even application networking So go check that out But we're gonna dive right into it you might have heard of something called ambient mesh which was announced last year as a joint effort between solo and Google and What it really is is a new mode of operation Within Istio or within the Istio service mesh, which offers up a sidecarless approach So within Istio today You would normally deploy something called a sidecar resource alongside your your main application container because it behaves as the proxy to process requests process policies Provide a level of telemetry and whatnot, but there are certain Applications out there that don't really benefit from it and actually are hurt by it So for example sometimes you may be experiencing things like a race condition between one of your applications trying to come online Before a sidecar comes online, so you can't quite get Your application up and on and online so you have to tune this and set something like a there's a field called the hold Application timer within your deployment so that you can ensure that your sidecar comes up online first There's also some other elements like the server send first protocols like sequel and another you know data database style of Applications and they don't play nicely with the Istio service mesh with the sidecar or using any sort of sidecar for communications So this is where ambient mesh shines because it provides a an opportunity to simplify that operation altogether What that means is while you still get the the benefits and the capabilities of a service mesh The function of the sidecar shifts away from the actual application and moves towards two components We've actually broken this down into layer four and layer seven So the two components are known as the z-tunnel and the waypoint proxy that Z-tunnel what we're really here to talk about the zero-trust tunnel is what's doing a lot of the activity for Tunneling requests between one particular service over to another service maybe within the same node or another node now The reality here is that You know not all applications are going to require the sidecar some will require the sidecar But not only that what happens when you need some sort of layer seven style of policy So this is where that waypoint proxy comes in no effect here to basically come in and say let's Act on that policy whenever there is one that comes about we'll talk a little bit more about how that all flows but the mode of operation changes here the direction of traffic changes and This means that we can provide Istio or service mesh like capabilities for workloads that don't need the sidecar Whereas we can also provide it for workloads that do take advantage of the sidecar There's also some other interesting components around how we route traffic within let's say an ambient mesh environment So we deploy something called a CNI node pod that is Deployed as a daemon set alongside the z-tunnel the z-tunnel is also deployed as a daemon set Now if you've ever worked with IP sec before there's something interesting that goes on you have to define an access control List that tells the tunnel. Hey, this is the traffic. We're gonna encrypt This is the source traffic. We're gonna encrypt. This is how we also operate in the world of ambient mesh We have traffic that is tagged towards the z-tunnel pod that's gonna tunnel over to another node and Once we've identified what that traffic is it's encrypted. It's sent across the tunnel and it gets to its destination So the same process flows for when we're having to do this with let's say a waypoint proxy But we'll talk a little bit more about that shortly But when we look at it at a deeper level, we have two services service a and service be when I have traffic That's Dustin from service a to service be that traffic is gonna go towards the z-tunnel Which used to be the sidecar that z-tunnel will actually derive Identity from that workload and use it to form its end of MTLS The destination does the exact same. So now you have end-to-end MTLS between these z-tunnel pods This way when your traffic or your requests are going through they're gonna still maintain their their encryption all the way through to the destination now Actually, one other thing I will point out though is that this is strictly layer 4 So if I'm just doing strictly or for communications TCP or UDP style of communications Then I only need the z-tunnel object or the the z-tunnel pod that gets deployed per node when I need When I need layer 7 authorization, for example, this is where the waypoint proxy comes in We deploy this at the destination side so that as traffic is coming in or as layer 7 requests are coming in That waypoint proxy is supposed to sit there and analyze those requests And then we'll allow that request to flow through or deny it altogether or provide some sort of response to the originator Now the other interesting thing though about the waypoint proxy is that it actually makes use of the Gateway API spec in Kubernetes So you specify a gateway class which I think Jim is going to show you a little bit later on when we get into the demo and That acts as our enforcement point for layer 7 Now one other interesting thing about this is that the waypoint proxy still uses Envoy because Envoy is the most Performant layer 7 proxy that we know of and that we've used alongside Istio for a very very long time Now what are the benefits of using? Ambient mesh so you cut back on race conditions like I mentioned earlier on but then you also have to Think about this this idea of sidecar injection it no longer is there Which means that you can easily onboard services into a mesh without having to roll them out again and restart that deployment The other thing too is when you have to upgrade let's say a sidecar or now more specifically a z-tunnel You're not having to do this on a per pod basis. You just do this at the z-tunnel level So that way you're cutting back on your overall upgrade parameters in terms of how many objects you have to upgrade The other thing too is you're also limiting your blast radius overall in terms of who gets affected in terms of your configuration Let's just say you and you applied a configuration that accidentally applied to all of your sidecars now all of a sudden You've locked out all of your services and they cannot communicate with each other that could that could actually be thwarted with Something like ambient mesh and using the z-tunnel The other thing too is with respect to cost savings and resource consumption Because you're not deploying sidecar resources everywhere and only where you need it You're now consuming less CPU and memory and even not having to worry about the additional configuration that goes alongside those services and Then this all of this actually works to contribute to providing that zero trust network Now let's take a quick look at Things like how do we derive identity and what actually goes on with this tunneling? So like I previously showed you I have a service service a or app a that's trying to communicate with app B over on the right hand side Now we already know that because these two workloads are tagged for ambient based traffic or ambient based workloads We already know to put that traffic or that request into the z-tunnel and then encrypt it with MTLS The other thing though is that you know, what is this tunnel? Like where did this come tunnel come from? So there was a lot of engineering effort that went in to create this new overlay called H bone or the HTTP based overlay network encapsulation and what this provides is a TCP over HTTP mechanism so that you can carry your your requests over this Overlay network or this overlay tunnel and this is occurring from z-tunnel pod to z-tunnel pod So remember we deploy a z-tunnel pod at a daemon set level So if you're no sorry if your cluster your kubernetes cluster has you know ten nodes Each one of those nodes is going to get a z-tunnel pod Which means that each one of those z-tunnel pods is going to tunnel to each other when there is traffic going over that wire Now there's the element of identity Now the z-tunnel itself actually impersonates the identity of that workload So if app a is trying to communicate with app B Normally if we had the sidecar the sidecar itself would assume the identity Using that spiffy ID format and then you know would form its mtls And this would happen on the other end as well in the case of z-tunnel that Identity is actually derived from the actual workload itself. So the z-tunnel will impersonate that workload for that request flow Now this is also where we can apply our layer for authorization policy So if you're very familiar with how you can work with firewalls I look at this as more of a layer four layer three firewall because now I could just go ahead and say I'm gonna a certain set of Workloads have this label. We're really looking at it at a TCP IP layer We're not looking at it at any you know at layer seven So we can implement a kind of like a firewall policy to say this service can't talk to that service And we could do this repeatedly if we wanted to Now how does z-tunnel actually get the certificate? So z-tunnel will actually solicit the workloads for its service account token once it gets that it actually goes to Istio D the Istio D control plane and says hey I am I am actually a z-tunnel on this node and I'm trying to impersonate or I'm trying to Service this identity for this particular workload. Can you run through the certificate signing request for me and issue me a Certificate and say we're all good to go and once Istio D is able to authorize that and because of all the trust That's already built in says sure. Let's go ahead and do it It'll issue its identity for that particular workload and it'll do so for the other end and then we'll proceed to create that mTLS end-to-end or that mTLS encryption end-to-end Now if we have multiple workloads that same z-tunnel pod is going to service all of these different workloads that exist on Let's say that node and this you know, you might wonder okay What happens when we have many many kinds of workloads running well There are different ways to scale that we're investigating one thing I will point out Istio ambient mesh is still experimental It's not ready for production, but the reason I say that is because we're still testing. We're still scale testing We're still validating various kinds of use cases that we can achieve with with ambient mesh So in the failure scenario here, let's say that this z-tunnel pod went down What would happen? Well Kubernetes will try to reconcile that but why did that pod go down? Did the node go down did the node get disconnected from the actual network? There are a lot of different reasons and varying failure conditions that could have contributed to that failure now How am I doing for time I? Just want to make sure okay. Alright, cuz I got to make sure I gave Jim his time for his demo now in the situation Where I want to implement layer 7 authorization policies I have to deploy a waypoint proxy like I mentioned previous previously before what that means is I have to deploy a Gateway API resource Specifically for the waypoint proxy for the destination workload where I'm trying to communicate towards and then I'll go about creating my layer 7 authorization policy when I create that policy it could be you know Service a or app a can only communicate and make HTTB get requests towards app B But not delete requests and as long as that policy is put into you know into effect and and deployed What ends up happening is that z-tunnel pod on the source of that traffic? Will encrypt using mtls towards the waypoint proxy the waypoint proxy will derive the identity of the destination workload much like it Would do for the z-tunnel and then it would tunnel to the z-tunnels on the destination node And then that z-tunnel will unencrypt the traffic and then directed towards that final workload So the request flow changes a little bit because now you're having to go through another proxy But that's just because we want to enforce layer 7 There's three conditions that I want to talk about in terms of what happens when I have let's say an app compromise Let's say a proxy compromise or even a node compromise So app compromises can come from a variety of different scenarios Hey, I pulled down a container that I didn't realize was malicious and now it's in my environment That can contribute to your app compromise So maybe you need to be thinking about, you know, how do I prevent us from pulling down Containers that we shouldn't be pulling down and deploying into our environment when it comes to When it comes to the proxy itself The original sidecar proxy has so many different Ways to you know create policy ways to tune it there are a lot of different observability filters that you can tap into and This is great for the sidecar because it provides all of that functionality But one of the the biggest challenges here is these are all the different Points of attack or points of potential vulnerability. Now if we slim down that proxy to just Doing layer 4 and MTLS then we can remove all the additional features and we can just secure that surface area Significantly, so this way we have a smaller exposure area and we limit ourselves to you know, the attack surface all together And then there's the final piece of node compromise So node compromise is interesting because I get that question a lot. All right I have an app that's going to be talking to the z-tunnel to you know forward requests over that Communication between let's say that pod and that z-tunnel pod is not encrypted. Okay, that's great That's that's that's wonderful, but here's the other thing. Why does someone have access to your node? Why do they have root access? You know, this isn't this is more of a physical control situation This is more of a role-based access control situation Do you have those proper policies in place to prevent people from being able to even get into your node? Are you using secure operating systems or even secure distributions to run your clusters? So there are other considerations to think about other than the fact that app a and the z-tunnel don't have encrypted communications If someone gets you know access to your nodes, you're in a lot more trouble than what we're seeing here and With that that actually gives me the opportunity to pass it over to Jim to do that little demo. Thanks Jim All right. Thank you, Marino So I don't know about you, but I'm someone who really enjoys not just hearing about how things work But actually seeing how they work and so we're going to we're gonna do that here with with a brief demonstration of of ambient mesh in the in the time we have remaining and So we're actually going to start from the from the basically from the ground up Okay, we've done a little bit of pre installation here You can see we have a we have a kind cluster has four nodes on it there's a control plane node there are three worker nodes that are that that are active here and So what we've done is we've we've There's actually a script that goes along with this if you're interested in in learning more you can actually follow along with this Exercise and and do a lot more than just what we're going to accomplish here in the next 10 or 15 minutes By going to academy dot solo dot IO look for this ambient mesh course right here. It's all perfectly free and You can actually go through this exercise on your own time and and dive into it a little more Deeply so at this point. We have a we have a four node cluster that's here. We've deployed that And we have also we have also deployed Istio now at this point ambient is not the default mode for Istio was As Marino said ambient is still is still experimental It's one of the top priorities for the Istio community this year to to get it hardened and get it production ready But at this point it's not so so we've actually installed the ambient profile In in Istio which it gives us it gives us the the Istio D Control plane kind of core Istio capabilities It gives us an ingress gateway and also the CNI components that that that Marino mentioned that's responsible for redirecting traffic between the the worker the workloads and the and the Z tunnels that actually manage the layer for policy Policy enforcement. Okay, so we've we've done that. We've done that installation You can actually see the The pods that are that are available here. So here are the Here the Istio system pods you can see there's the Istio CNI node. It's deployed as a demon set So you can see we have one per node We also have the Z tunnel components that have been deployed as well now at this point They're not really doing anything because we have an activated ambient for any of our application workspaces But they're there. There's one deployed per node. They're ready to they're ready to go And you can also see we have deployed some workloads that we're going to be using in this exercise so to To accomplish that we're we have a little simulation of an e-commerce system You can see this diagram at the right. There are three components here. There's a web API There's a recommendation service and there is a purchase history service. Now these services aren't really real We're using a little framework called the fake service that allows us to just basically Test policy enforcement against the the communication as it happens across Across the different service boundaries and basically when you hit each service What you're going to get back is just a kind of an echo message that says hey You're in the purchase history service now and that sort of thing so that's That's our that's our application workload if you will we also have a couple of client pods that we have deployed here There's one call sleep and one call not sleep. You'll you'll see those be activated in just a bit Okay, we have also we're also using just standard Istio APIs to expose These workloads to the outside world if you're familiar with Istio, there's nothing there's nothing exotic or Unusual here at all. We're just using we basically have a we have a gateway component That we've defined that just says hey We're listening in on the host Istio explain.io on port 80 with HTTP And then we're also establishing a virtual service to route traffic It's intended for that domain to our web API Service which is kind of the the head of our the head of our application chain That we are using here Okay, so once we have once we have all of that in place We can do just some simple tests here So this is one. We're actually going to go through the through the gateway Through the Istio ingress gateway coming from outside the cluster It's going to route to the web API service which is in turn going to invoke all of the others In the chain and you can see that when we when we call that here There is our web API Response and you can see it makes some upstream calls one of those is the recommendation service the recommendation service in turn Makes a call upstream to the to the purchase history service So a pretty simple setup everything is working We have ambient installed on the cluster, but we've not really activated it yet And so that's what we're going to do Next so we will check that out everything should be good and We will move on All right to the next step so we're going to do a couple of things here first of all We're going to add services to the ambient mesh. You're going to see how the activation Takes place. We're going to look at some specific layer for authorization Policies just basic things that you can do that are a prop that are applied via the Z-tunnel Component and then finally we'll show you just a really quick example of a layer 7 Authorization policy as well. So let's let's kick off This this exercise and the first thing we want to do so we have the ambient profile of Istio installed But we've not activated it For any of our namespaces yet So what we're going to do we're going to label the default namespace which is where our web API Purchase history our e-commerce application Services are deployed and so once we label that what's going to happen is we're going to see We're going to see ambient mode be activated for the services in this in this workspace Specifically you can see that by taking a look at the logs of the CNI node We won't go into any detail here, but you can see it's recognized. Hey, we're setting up this default namespace You can see You can see where some routing rules are being defined here And that they're routes that are being added from the from the the source workloads and the target workloads to Their local their local z-tunnels, okay So the nice thing is we're not having to go and restart those components at all We're not having to inject A sidecar or anything like that, but you can see here for our application pods That we still have a single, you know, there's no sidecar injected There's there's no there aren't two containers that are active within those within those pods There's only a single one and in fact the z-tunnel is handling the The routing of the traffic, okay, so so once we have Established that kind of verified that the the CNI has is redirecting our our traffic Actually what we're what we're going to do is you'll see the What what what will happen is the traffic is going to look something like what's depicted in that diagram that It's very similar to what marino showed where we're gonna have a client app talking to its local z-tunnel it's going to establish an H-bone tunnel to a z-tunnel on the target machine and Then it will route traffic to the actual target services. Okay, so No sidecar is required. So if we generate some traffic now to this To this API service you can actually see what the z-tunnel is Is doing so if we take a look at the at the z-tunnel logs You can see here's here's the the HTTP connect tunnel that's that's being established here You can see some of the the spiffy IDs that indicate things like for example We're going from the you know like from the from the web API to the recommendation engine You can also see here at the very bottom a record where that we're capturing Outbound traffic and there's no waypoint proxy that is that's involved And that's because all we're doing here is we're just activating the layer for Component of the network right we're encrypting the traffic We're using we're using cryptographic identities to establish the connection between between the z-tunnels and the and the source the source and destination services But because there's no layer seven Policy that's being applied. We don't have to we don't have to have one of those Activated at this point so that's an important consideration for a lot of what we see customers who are adopting ambient mesh or adopting Istio they care primarily about the the security aspects of Securing the services within their mesh they don't necessarily need the full layer seven capabilities And so this gives you a way to Incrementally adopt just the components of Istio that you need without biting off the full The full layer seven model, okay, so we can see the traffic now going through the Through the z-tunnel and now let's actually Apply some policies to that to that traffic. So first we'll start with you know good zero trust foundation here We're gonna start with a policy that basically denies all Denies all traffic. Okay, so this is an allow Nothing authorization policy another thing you might notice is that the the Istio API's we're using standard Istio API's here The API's themselves have not Changed it's simply the way the Istio decontrol plane is is activating these policies is by programming You know z-tunnels and waypoint proxies as opposed to programming sidecars that are attached to each individual workload So once we establish that allow nothing authorization policy, we'll do the exact same operation We did before that worked fine and not surprisingly what's gonna happen is that request is going to be denied So we will expand out our authorization policy again using best, you know least privilege access sort of Practices will establish a suite of policies that basically authorize each link in that service chain So here is one for the for the web API component. It's going to allow It's going to allow requests that come from either this this sleep identity the sleep pod or from the Istio Ingress Gateway Component all right So and then in turn the next link in the chain the recommendation service is only going to allow Access from the web API Identity and finally the third link in the chain the purchase history component is only going to allow access from the recommendation service Okay, pretty straightforward stuff, but gives you a nice policy-based foundation for For establishing this sort of thing now. So now what do we expect to happen if we use that same? That same invocation again in this case Now we expect that it's going to work just fine because we've authorized each link in the in the service chain And in fact, that's exactly what you see. We're hitting web API We're hitting recommendation and we're hitting purchase history. Okay Now one final step here, and then I'll turn it back over to Marino to wrap up So We've talked about we've talked about the layer four policies how those are applied via the via the z-tunnel We can also apply layer seven policies for services for identities where that makes sense We can spin up the additional Infrastructure to to make that happen But we don't have to bite off on establishing that infrastructure for each and every Workload in the mesh. So in this case what we're going to do is we're going to establish a layer seven policy for this web API Identity okay, and so to do that we're going to first spin up a waypoint proxy Which is at this point just it's an envoy proxy just like just like the like sidecars are in Istio that It's actually going to be the policy enforcement point for for this policy. So all right What does the policy look like we're going to do well first of all, let's actually as Marino said We're going to use the gateway API to actually spin up that that waypoint proxy so you can see this gateway This gateway corresponds to the web API identity service account here, so let's Connection close the demo gods have struck I hate it when that happens All right Here we go. Did you offer your sacrifice to the demo gods this morning Marino? I forgot so that's that's probably what's going on here all right, so So all right, so we have we have established that gateway and If the gods are being kind to us we can look now and we should see that a Waypoint proxy corresponding to this web API service has indeed Been been spun up for us and now let's actually do something interesting so so far We've only been specifying policies that require layer for access Let's do something that that introduces a layer seven component an HTTP component So we're going to expand out the web API Policy so that now We're using we're allowing traffic from the same principles as before but we're only supporting HTTP get methods All right, so something that requires layer seven access to to be able to enforce So when we when we activate that policy now if we try to do the same thing that we did before This is going to work because we're issuing an HTTP get Against the against the web API. All right, and so that's all that's all fine. That works but if we Try to issue a delete request instead of a get request What do you expect to happen the policy is going to kick in and we're going to get an access denied? Message there you can also see we get back a because this is at the HTTP level. We also get back a nice HTTP 403 Forbidden code as well. So with that I'm going to shut down the demo and turn it back over to Marino to take us home All right working perfect. Thank you so much for the demo. Appreciate that Jim. Okay, so let's go back to the Is it I got hidden somewhere? Let me take you back There you go get play up there Nice, okay So just to recap Istio offer or Istio ambient mesh offers you a new mode of operation For a side carless operation So the the fundamental difference here is that when you roll out services into your mesh with ambient mesh You're not having to deploy a sidecar, which means you're not Injecting you're not having to modify the existing workload. You're not having to restart it You actually tag your your namespace that you want it to be a part of the ambient mesh Environment and once you do so then the way point sorry then the z tunnels get deployed and then they start Looking for traffic destined towards other ambient mesh based workloads Now what this means is because we're you know operating with less sidecars that means there are less Resources consumed, but does that mean we lose any capability? No, because we've shifted that operation towards the z tunnel and the waypoint proxy if I want layer for authorization It's there layer seven. It's also there, but through that gateway API deployed waypoint proxy Finally, I wanted to just point out one very interesting thing here You might have noticed that the waypoint proxy as well as the z tunnel currently use implementations of envoy There is work being done today in the Istio community to change that that z tunnel proxy to be rust-based So that it could be far more performant It could also allow us to you know scale in different ways as well So you could go check that out There's actually if I'm not mistaken the Istio community has even got a branch for that that you can go check out and even test it Up for yourself to understand like what the performance might look like in fact I think one of the Istio contributors also created a very good document that just walks through the different Envoy a go-based proxy and a rust-based proxy and the different performance Metrics that come out of that as well Finally, I will say if you are if you are interested to play with ambient mesh in detail We have a lab that's available through solo Academy. Go check that out. It's academy.solo.io We have ambient ambient mesh based workshops. We also have Istio We have ebpf and psyllium so that you can learn the whole stack for application networking Also, if you want to continue that conversation about ambient mesh You have more questions if we're not able to answer them here obviously come visit us at G 18 We're we're in that expo hall somewhere, but you can't miss us so Let's open it up to questions and by the way We also have giveaways the ambient mesh book so we can give you some more details If you're if you have a plane ride home like I do this is great You can read it. You can read on the plane on the way home. So yeah come come ask us for one All right any questions Yes, go ahead So currently as it stands you can't at the moment only because there hasn't been any sort of I'd say Configuration or coding or programming for let's say the Istio see an iPod to direct that directly to the waypoint And in fact the waypoint is only meant to be deployed when you have to you know use layers of an authorization Most of the trip are all of the traffic should be flowing through the z-tunnel so that it can decide Do we want a tunnel towards a waypoint or directly to another z-tunnel so that we can Well, yeah, there could be other options specifically for Istio We've gone on the path of using the z-tunnel and then taking advantage of the each bone protocol that in fact if you go look at I think Istio one dot one six now supports if I'm not mistaken Don't quote me on this because I gotta go back and check now supports each bone to the sidecar as well so this means that There's more integration going towards this ambient mesh and non ambient based workloads that run the sidecar So the potential of you saying when you just mentioned hey, can we just go to the the waypoint? That could be a possibility and that's something that we could probably feed back to the Istio community to further develop So another question over here Intermediary yeah, that's right Okay, so my what I would say is we're not there yet. Well the Istio community is not there yet But that's definitely being investigated If we could do it with Istio today, I mean there's no reason we can do it with ambient mesh as well It's just we haven't gone down that path Envoy's SDS Yeah, I don't think that's available not today in the community yet. Yeah Any other questions? Awesome. Well, thank you very much for coming by appreciate your time. Thank you. Take care