 Okay. Thank you for being here so late. This is the normal exit announcement. You should already know about it. And this is my talk. How CF Networking could solve your environment-specific use case. Before I go into about me, my name is Konstantinos Karaboyas. I'm working for SWISCO. And the last two years I have been actively involved in the open source Cloud Foundry team. First, I started doing my dojo in the CF Networking team, where I got familiar with a pre-silk version of it. Then I joined the current team in London remotely, where I got some good knowledge about how containers are working. And now I'm part of the CFCR team, remote member in Dublin. And this is my Github and my Twitter. The agenda of our presentation is straightforward. We are going to discuss about our problem that we need to solve. Then we are going to discuss the two things that make it possible to give our solution. And this is the CF Networking boss release and the change CNI plugin feature. We are going through then the custom stack of the prototype. We will do a demo. We will conclude and we will have time for some questions. So, the problem that we are going to solve is called external traffic identification. According to it, if two apps are running in the same Diego cell, we cannot and these apps are trying to speak to an external service. The service owner cannot identify the traffic. Cannot say that this traffic comes from a production app, or this traffic is coming from a non-production app. Why this is happening is happening, because if we see the traffic inside the Diego cell, we can see that there are two source IP addresses and it's easy to map. But once we go outside the Diego cell, the traffic has been masked. So, we now see only the external IP of the Diego cell. Of course, when we go outside the operator, yet away, same stuff. This time the external IP is a public IP of the operator. So, it goes to the gateway in the service where we say what's happening. How can we actually put a rule that it will say, okay, I want to allow traffic only from my production app. This is also the requirement from the customer that he wants to put a traditional firewall rule. And in order to do that, we need actually to split the traffic all the way through from the source. Basically, we need to be able to put in our gateway, in the operator gateway another rule that it will say that if I have a production app, please put this source IP address. And then it goes to the question, how do we do that in our network, in the custom operator network? And this is the custom behavior that you want to add. This behavior probably will not make sense to do it in the upstream. It will not be accepted. So, to show, I decided that I would like to solve this problem by simply controlling the source port range from the traffic. So, I want to have a rule that it will say, if I have a production app, put this specific source port range. This will work because then it will be easy in the gateway to put if this source port rate 5000 to 5100, put this production IP and then advertise this production IP to the customer and the customer will put the rule. So, now the question is, how do I do that in the CF platform? Before I will present my solution, this kind of alternative that I have discussed over to the last year have two separate safe deployments. This might make sense if we have quite a large customer. Alternately, it will have isolation segments by having only specific subset of the Deco cell running the production app. We could just differentiate the IP, but resource-wise might not be optimal. From the network perspective, probably this is a better solution, which says that I'm going to put the operator gateway inside the container network. But given that this gateway might be an expensive hardware device, this is not very applicable. And there is also, of course, the service mesh that is going to solve everything and this is correct, but this will make the assumption that it will open up and it will have our customer to trust this. Plus, this is a very simple use case to actually try to introduce service mesh just to solve that. Two things happened that we could address the problem with another way. The CF networking stack changed. The CF networking team replaced the old Kawasaki stack and on one side enabled the direct communication so make the microservice experience better, low latency. But in my context, what it basically did, it added a CNI plug-up endpoint in the platform. So, it added one component that is called Gardner-Lexterl Networker and this is part with Gardner-Lexterl the runtime. It put an interface that is the CNI interface and it added a better included solution that is the red part here. So, and said that by having the ability to provide a plug-in that is according to the CNI standard, then you can actually replace and put your stuff here. A bit about the container-network interface. This is the interface that is happening between the runtime and the net plug-in. So, it tries to standardize the network implementation and make the code base of the runtime smaller, cleaner and allow more network venture to add a plug-in. It seems because it needs only one config file that is fitted to the runtime then with some runtime environments, there is just a call in a standard call in the plug-in and we set up the network. CNI is not what solved my use case, it is a new feature that is called chaining. So, now it allows actually to plug to call more than one plug-in. Let's see a bit how it works, just as a refreshment. We start by having one configuration file. We feed this configuration file to the runtime. The runtime then creates the container during the run COCI call, gets back the network name space. This is only what it needs. Then it's going to create a STD that it will feed to the binary, that it will take the configuration file and the runtime information. The most important is the network name space and it's going to call the CNI plug-in. The CNI plug-in is going to do what it needs to do to connect the container, the network name space into the network and it's going to return. But now, in the chaining feature, this changes. So, instead of having one configuration file, now basically have a conflict list. So, it's saying that you're going to call three plug-ins with the reconfiguration. So, the first step stays as it is. But now, instead of exiting, the runtime is going to take the previous results from the first call and it's going to call the second plug-in. It will get the result back, called the third plug-in. So, the network stack. For the version that I started, when I started working on that, the Gardner and L'Extension Networker did not work out of the box. I had to modify it. But the changes that I did were minimal. So, basically instead of calling the conf from the Stadra Library, it's to call the conf list. And this was all what I had to do. Then, I started building my chain. So, I put first one binary that is called bridge that takes care of the connectivity. This is an open source binary maintained by the container networking team. On the right, you can see the configuration file that this plug-in requires. Then, I added another binary, that is the IPMAS, the port map. This is also an open source. This is the plug-in that basically controls all the traffic that flows in the container. It's how I'm going to access the container, the process that is running inside the container. On the right is the configuration file that needs to run. Here you can see that there is something that is capabilities. Port mapping equals true. So, what does it mean? It means that during the runtime, the runtime should expand this and provide some more information to the plug-in. And the case is like which host port in the Diego cell to bind, which internal port to the container and which port to call. By having done that, I keep the CF push working. Then, I start adding my own stuff. So, I wrote a small plug-in that I called IPMASC. So, the intention of this is to control the traffic that flows from the container outside. Here is the configuration file that needs. It needs this time something different that is called mask entries. And during runtime, this has been expanded to give me the source port range that I want to attach to this container. In this moment, it became clear that it's very easy to add behavior. So, I said, okay, let's experiment a bit more. So, I did another plug-in that I called IPRoot. So, this is what it's doing, it's basically controlling the routing table of the container. And again, this in order to work, it has to feed some information. Roots. What I have in mind when I was writing this plug-in is if I can replace the application security groups to use the routing table instead of IP tapes for performance reason. I keep going, so I just create another binary that I called DNS register. So, what this is doing is just taking the metadata of the container, the application ID and then register itself to the console that already exists in the Cloud Foundry. And because it was a bit hard to debug stuff, I wrote one other plug-in that basically what it's doing is just writing in the disk all the configuration file. So, it was easy for me to debug this black box. A such plug-in might be interesting also for auditing reason. And, yes, I took that, I created a custom boss release which basically deploys the Garden Excel Networker, all my binaries, my configuration files and does some pre-steps. And then I just add another ops file which basically saying just enable my CF extended networking. The demo. The demo, I could not do it in my boss light because basically I could not attach two public IPs to my laptop. So, I picked AWS. So, I did just a bubble app, I deploy CF, I register my DNS domain and then manually, I had to add two external IP address in the NAT instance. In the NAT instance that we see, from where all the traffic goes out of the container. And then on the other side, I just built one MariaDB service in a server where I control the IP tab rules. And then I just deployed the CF external networking. Now, I can do the small demo. So, we see here that I have the boss VMs, I have two Diego cells, I have login in my Cloud Foundry and I have two spaces. One dev and one production. The first thing that I have to do is basically to create the mapping. So, to say that this production due it should be mapped to this port range, 5000, 5100. I do that and I have to do that also for the dev space. So, the dev space will take a different port range. Once I do that, I basically have to deploy. So, I'm not going to deploy for timing reasons, but it's just a call that I'm using in my ops file and I'm just doing a boss deploy. And the ops file is just replacing this, adding my custom boss release. And yes, when I will deploy, I could just go to the ego cell to show you what exactly happened. So, it added the binaries that I'm going to use. This is all the binaries that I described before. It added the configuration, the mapping. So, jobs. Config mass rule. So, you see here that this thing, which page will take which source port range. And there is also the configuration file that is exactly what I presented in the slides before. So, now I'm ready to deploy and up. So, basically I'm going to do... I'm going to the production. Oops, sorry. Prod. In the prod, then I'm going to do a push prod. I'm going to do a no start, because before, I have to do, to set up an environment variable, a variable where to actually connect to the external service. And then I can just start it. This should take a bit of time. And then I have... Yes, so it seems that... I have a history source. But I have another app that is running, that is called prod app. And it's in this space. And I can also see that I have in the dev space another app that is called dev app. So, if I do a care to the dev app, I should see that it's forbidden. And if I do the same call to the prod app, I should see that again it's failing. But this time I can see that my IP is different from this case. So now I have to take this IP and just give it to my SQL operator, to basically the network guy that fix the rules. So, what it has to be done basically to add a rule that is going to allow the traffic from only one IP address. So, when I do that, and I do the care from here, I can see that now there is a connection succeeded and I can get the result back from the MariaDB. While the other one still is failing. And the same stuff I can do if I click here also to see it in there. Dev app fails, prod app succeeds. So, this probably is very useful because almost all the container based orchestrator have this problem. And, what happened? Okay. So, one thing that I want also to show is if I do NSH, CFS, PROD, CFA, CFS, SH, PROD app and I'm inside here and I can do the IP route. I can see that basically I have control the routing table. This one entry that I just put inside. And if I do the same here, but in the dev space and I do NSH dev app, then I can also see that the DNS registration works. So, basically if I take this and then I do NSLOOKUP, dotService, dotCF, dotInternal, be able to see that I have registered and I have provided the NSNames and the ping also still works. So, before I conclude, I would like just to show you how things actually are implemented in the Diego self site. So, in all, everything takes place in this post routing. Tough nut. Where basically I'm adding my rule in front of everything else and then I add all the information that I need here. With saying which port range will take. So, this is very flexible because at the end of the day I can put whatever kind of IP service rules I want that to work only with a specific container. And if I see also the, we got that CNI, the net part, I can see how it looks like they actually call. So, this is basically how the runtime is extending stuff. So, from the configuration file that was the capability, this port mapping equals true. The runtime has to expand it and give the actual information. To go back to our presentation, I don't need the recording one. So, we have to discuss a bit the user interface. In the demo there were two things that I did not like. And this is the fact that I had to do a boss deploy in order to add the mapping. The way to do it. And also I had to modify the code of the CF, of the garden network external networker because I had to do this small task like take this map and take my space ID and find which is my source port range. In order to produce finally what the plugin needs to work. And this is not ideal because this information should not be relevant to the platform. It is only a contract between the plugin and the operator or the user. So, as a user interface in my use case would have been to have a command like that for example. That I will do a CF, set space metadata, see an arcs and then whatever I want. That I know that this information will pass and go in the plugin and will be interpreted correctly. In the future I would like to see the garden, the CF networking team to adapt a native chain CNI and I think they have already started looking at it. I would like to see to replace the bridge with a silk because this will allow us to keep all the policy framework and everything as it is working now. And I will be curious how we will be able to change the behavior as far as the traffic goes out, the routing table and the traffic goes in, in real time. Because when we are using CNI it's only about the calls that we are doing when we create or we delete the container. It does not have to do with the real time. So, if we implement something there we have always to restate our apps. And this is not ideal. And of course to investigate how these things could are working out of the box in a Kubernetes cluster. Here are some links. One is one talk from the previous summit where I explained more about the CNI. The second link is a block that explain a bit more details how a runtime is using the CNI and the chaining one. And the source code of my demo. In the first one is all the CNI plugins that I used and the second one it's the boss release. Probably it's not going to work out of the box because of the blobs but it should be good enough. Yes, and that was my talk. Thank you very much.