 Hello, my name is Ted Ross. I'm a senior principal software engineer with Red Hat out of Westford, Massachusetts. I've spent most of my career working in embedded data networking systems and network security. More recently, I've spent a lot of my time working on the more abstract notions of communication, like messaging and such. Today, what I'm going to talk to you about is some challenges with Cloud Native Interconnect. I'm going to introduce the notion of a virtual application network, and I'm going to do a demonstration of the Scupper project. Some terms before we get into the meat of everything here, that I'm going to be using just to make sure that you understand. First of all is virtual application network. Sometimes I will refer to this as a van. This is really about multi-site hybrid edge Cloud Interconnect. It's about connecting processes and services together, no matter where they are, even if they are on multiple sites or multiple clusters. It is a higher abstraction for process interconnect than, for example, TCPIP. Scupper is an open-source project, and it is an implementation of a virtual application network. This is what my team at Red Hat is working on presently. Let's talk a little bit about internet and how it's structured. The internet is divided into two pieces. There's the public internet, which is a public addressing space. Everything in this space can be addressed from anywhere on the globe on the internet. There are private-edge networks as well, and of this, there are literally hundreds of millions of overlapping private-edge networks. They use the same address space overlapping. This is the reason why the internet scales as hugely as it does. The typical pattern of communication then that you see is this vertical communication, where the clients in private-edge networks use servers that are in the public internet. That's a generalization, but this is the way that the internet is basically structured. What this means is that any communication that occurs between endpoints in the edges with each other is actually an illusion because that traffic is actually going to a server and then back down again. That's the way that those applications are actually written. That's why I'm going to claim that the internet and client-server architecture go hand in hand. They both grew up together, and the internet is very well suited for client-server architecture. I will further claim that cloud-native architecture is not client-server architecture. Cloud-native wants to do things much more flexibly. We want to be able to talk horizontally east-west, north-south, and every such direction that might exist, and we don't want to be constrained necessarily by where things are located. Let me then also talk a little bit about addressing as background and as I'm sure you're all familiar with, TCP, IP, and internet use a host address plus a port. Now, we don't generally talk about numeric host addresses, we actually use names, but those names are then locally mapped to addresses using a name lookup service like domain name service. The name maps by DNS to a host address, the host address addresses a host, the port then says which process on that host and my wishing to communicate with. Now, by contrast, a van uses a service address. Same name, but in this case, the names are actually the address that are being used for routing across the network. The names reference a process rather than a host, so we're routing to processes rather than hosts, and the van is natively multi-access rather than unicast, and what this means is that it is expected that multiple services will be deployed with the same address. This is actually what we want as Cloud native developers. We almost never want to create a singleton instance of a service. We almost always want to have multiple instances of a service, and we do this because we want to have high availability in case one of them fails. We want to be able to handle high amount of load so that we can spread the load across different instances of the service, or we might be interested in locality, or we want a service in Asia Pacific to have a clients in Asia Pacific, and another service in Eastern Europe to handle services that come from Western Europe, and that makes for a better experience for everybody involved. Multi-access comes in two flavors as the unicast, which is about load balancing, as I've just been talking about, and there's multicast for distribution, so say for example, event distribution. If I wish to create an event but have that received by multiple consumers across my network, then I would use a multicast flavor of multi-access. Now, I should mention that vans, as I said before, are really about multi-cluster, multi-site interconnect, so this routing, whether it be any cast load balancing or multicast distribution is network-wide. It's not constrained to a LAN, it's not constrained to a data center, it's not constrained in any way by the underlying network. So if I want a multicast across the globe, that's how my application is set up that I'm going to be able to do that, and if I want load balance across the globe, I can do that as well. So a virtual application network as an abstraction is a layer that we can insert in between our application processes and the well-known TCPIP network. So it utilizes the strengths of the internet and doesn't ask the internet to do anything that it wasn't intended to do. So it uses that really strong connectivity and communication capability, but doesn't try to get it to do things like global multicast. The applications do not need to be modified, so if the application is written as an HTTP REST application, like I'm going to show you in my demonstration, or if it's GRPC, or if it uses some messaging mechanism, or even if it's like using just a home-brewed TCP or UDP protocol or a standard one like JDBC, it can run across the van unmodified. And the van is a natural fit into container platforms like Kubernetes, Docker or other things like Eclipse, IOFOG because of the way that they very flexibly handle local networking. So one of the important aspects of a virtual application network is the independence, orthogonality. So the shape of the application and the shape of the network are not related and they are independent from each other. So as DevOps, we want to allocate our services to site for our business reasons. We don't want to do it because the network is constrained or demands that we do it a certain way. So for example, if we want to do test and development of a subsystem or maybe our entire system, we want to use a small footprint for that. If we were deploying, we want to add regions because we've started advertising in APAC and we want to put up a front end in Singapore or we can do that. If we want to deploy our services on different public availability zones to protect ourselves from disappearing off the face of the earth if one of them fails for high availability. If we want to be able to scale our compute capacity with the changes in demand, the offered load, the demand, we want to do that so that we can be efficient with the cost. We don't want to over allocate services but we want to have services on hand when we need them. We want our sensitive data to be located in our private data centers. We don't want to put them in public clouds. And in fact, it may be required by law for us to do this. So even though these data services may be needed by things outside, we don't want to locate them there. And then in terms of development, maybe we want to connect our laptop to a van so we can do development and debugging in context without having to actually use a container platform for that process which makes the development process much easier. So here is like a notional aspect of a van, what it might look like where I've got a number of public sites and I've got arrays of edge locations and maybe their storefronts, maybe their factories, maybe their warehouses, whatever it might be that is driving my application. And I've got a headquarters and they're all interconnected in a way that I designate. So a couple of notes about this topology. So the intersite connections that these arrows are showing you, they all use mutual TLS with a dedicated certificate authorities. This is very locked down from a security standpoint. So no sockets or ports are open to the public that don't use this MTLS. So those connections are going across the internet but they are protected. So the only time that this application would actually open a port to public is if they actually wish to do so, like to provide an address or a portal or a website, for example. No VPNs, no IPsec, no SDN, nothing but vanilla networking is used here. And furthermore, it's developer accessible. So as a developer, I don't need admin privileges, I don't need cluster admin privileges, I don't need anything elevated, I don't need to make any phone calls to set this up. All I need is access to namespaces in these locations and I can hook them together using OVN. The redundant paths are recommended because they provide resilience to failure. I don't need a full mesh, but I do wanna have redundancy in case there are failures somewhere in the network or a data centers go down. So what am I gonna show you in the demonstration? So I'm gonna show you a network of four Kubernetes clusters. I've got Minicube running on my laptop. I've got OC4, OpenShift version four running in AWS, this is actually in Europe. I've got Azure Kubernetes services running in US East and this is straight vanilla Kubernetes 117.11 deployment. And I've also got a remote system in a remote data center that's running an old version of OpenShift 3.11. So, and I'm gonna hook them up in a network. So what I'm gonna show you when I do this demonstration is a K-native service that I'm gonna deploy on the laptop in AWS. Now I'm gonna designate the laptop as being primary. This is the one I want to use by default, but AWS there is a secondary backup and it can also be used to handle load in case the laptop service becomes backed up. So it can load balance and pick up load. This is sometimes referred to as cloud bursting. So I'm gonna show edge-to-edge indirect connectivity because I'm gonna put my load generating client over here on remote and it's going to be able to use these services. One of which on the laptop is not directly accessible. It's gonna have to be a two hop proposition to get there. And the other thing I'm gonna show, as I mentioned before is multicluster load balancing because I'm gonna show that service being offered in the two places simultaneously. So let's get started. So what I'm showing you is this sort of my demonstration view on the upper left shows the topology and then I've got a command line terminal color coded for each location so you can see what's going on there. So the very first thing I do here is I, first of all, I have to get the scupper executable from the scupper project. But what I will then do is just issue this simple command. I'll say name and this one is AWS. So I'm just gonna do a scupper in it, site name AWS and I'll do the same thing here called Azure, I'll do the same thing here called remote. I'll do the same thing here, I'll call it laptop. But one of the things I'm doing on laptop is I'm enabling a console cause I wanna show you the scupper console on at it. So I'm turning off the console off for the purposes of this demonstration because I trust my audience not to be attempting to hack it while I'm doing the demonstration. So this now has enabled scupper. It's injected the router into my namespace, the scupper router that can be used to set up this network. So the first thing I need to do for my two public sites is create a, which called a connection token. So I'll create a connection token for AWS. And I'll create one for Azure. And what this means is this, this is all the information needed to make a secure NTLS connection to this location. So I've made, I've set up AWS and Azure to be connected to, I'm gonna secure copy those tokens over to my remote site because this is actually a remote connection. So I've transported those tokens securely over to my remote site so they can make a connection. And then I will establish my first connection between Azure and AWS. So scupper connect my AWS and configure it to make this connection. And the next thing I will do is I will configure both of my edge systems, my laptop and my remote to make redundant system connections in to each of the public locations. So let's see if I can get this to work, right? I'll say AWS, so let's connect for AWS and Azure. And then I will do the same in my remote AWS Azure and they are configured to connect and it runs. All right, so while things are stabilizing here, let me talk a little bit about this picture. You'll see that there are arrowheads depicted in these connections. And these arrowheads are significant only in that they depict the direction of the connection establishment. So AWS and Azure are public locations. They have public addresses. They are in the public side of the internet. Laptop and remote are in private networks. So there's no way to make a connection to them. They don't have public addresses. And I'm not setting up anything fancy here like a VPN. So what happens now is once those connections are established, the arrowheads are no longer significant because the connections are long lived, they're established, they're secure. And once they're in place, they operate bi-directionally. So this is my network now that is established between the four different locations. So while I've been talking, I'll give it a chance to stabilize. Let's see what happens here. I will ask my laptop to say scupper status. What's the status here? And it's gonna tell me that it is connected to three other sites, one indirectly. That's kind of what we expect, isn't it? Cause it's connected to three sites, but one of them doesn't have a direct connection to it. Now, if I were to ask Azure, the same question, scupper status, and it should get back to me, hopefully with something similar. Yes, it's connected to three other sites and none of them indirectly. So this is correct. Azure has a direct connection to all the other sites. Okay. So let's get the services on my mini-cube on my laptop. And I've got one called scupper controller. I'm gonna grab his external IP address. I'm gonna go to a web browser, I'm gonna web to this 48080 as it's exposed. This is the scupper console and it's connecting up. And if I go to the sites tab, this is gonna tell me what's actually happening in my network and it looks kind of familiar. So this is a good sign that what we've set up manually is actually what we got. So here we have a network. You saw the whole thing set up and it's all established and ready to go. So it's time to go on to the next step. And before I do, I'm just gonna do an OC get services here on my remote for reference. And I'm now going to, and you'll see that there's the low gens, the client, low generating client that I talked about. And there's a couple of services associated with the scupper deployment itself. So I'm now going to do a K apply dash F service. So I'm gonna apply this K native service, deploy it here on my laptop and I'm gonna apply a YAML for scupper, which is just going to annotate that service so it can be exposed by a scupper. I'm gonna do the same thing on AWS. So here I will say OC apply dash F service. You'll notice that the AWS being remote and in Europe runs quite a bit slower than OC apply dash scupper. So I'll catch up as it catches up. So while that's going, let me now go back to remote and look at the service list again. So in this case, okay, here's the new list. So I've got the same ones I had before, but there's a new one called greeting. So greeting is a service for which there is no pod backing it or deployment backing it locally, but this is actually what has been done by scupper. So scupper, I put these K native servers up on AWS and laptop and then I scupperized them and now I have services in all four locations that can be used to access them. So think of this as a proxy service that can get to that remote place. So if I were to come over and look at OC get pods, so I've got the greeting pod running. This comes up initially, but when it's not used K native will scale it to zero. K get pods, same here. And you'll see that since I did this one earlier this one has already scaled down. So there's no pod running here, even though this is the primary. So what I'm going to do now is I'm going to access my load gem service and I'm gonna set a load of one. So it's just a little curl command that sets the load. And when I do that, when I say load of one that means concurrency, that means one request in flight at a time. So sends a request and response comes back and sends another one. And so it's like a synchronous process running and doing repeated requests. And if I look at the pods now in my primary I see I've got a pod running for greeting. The ZMLL8, so let's see. I've got that running. Now I can look at the status from my lower generated client and I'll see that of the last 100 items 100% were served by ZMLL8. So that means that even though my client here it's here on remote, it's accessing the service deployed on laptop. And if I come back and get the pods here on AWS I presume we'll be that long gone. And it is, so because it's not needed. I set a cost of five, a threshold of five. So before it would fail over to AWS. So let's go have a look at the console again. Let me look at the deployments and you'll see that I've got a deployment in laptop that's serving a client in remote. That's all good. Let's increase the load. Let's take it up to 10 concurrent clients making requests at the same time. And when I do that I'm gonna come over the threshold. So I expect that in AWS I'm gonna see a pod spin up and there it is. So I will now go back to my status view and I'm gonna see that I am balancing my load between ZMLL8, which is on my laptop and the new one, which is H5FPH, that is what's running here. So I will see if there's a balance between these. So of the last 100 requests, 52% were handled locally on my laptop and 48% were handled out on AWS. So the balancing is not round robin, it's based on the actual completion rate of the services. So it balances dynamically to the services that are running fastest. And now if I go back to my console again, look at my deployments, I will see that they're being balanced between AWS and laptop. So the client doesn't see a difference, it doesn't know, it's just getting better service because it's being serviced by multiple instances. So if I were to come back here and reset my load back down to one, I will see when I look at my status that I'm now coming off that AWS and all my requests are now being handled again by the laptop, which is my designated primary. So this is now running at the lower load, I no longer need the resources of the public cloud, it's just using my resources here on mini queue. Now I will look at the pods again on AWS, it will still be here, but in a minute it won't because K-native will spin this back down to zero. And if I look here again, you'll see that the numbers are increasing on my laptop but they're not increasing on AWS because AWS is no longer handling load because it is not needed. I'll take one more look over here and see if the pod is still running but it'll take a minute to go down and be terminated. So what I'll mention right now is that if you go to scupper.io and look in the GitHub repository associated with it, there are examples in there. And there is an example in there that is basically the demo I'm showing you here for KubeCon NA 2020. So it has all the YAML, all the source code for everything that's being run in terms of my client and server and the instructions for doing all of this. So if you wish to replicate this experiment and demonstration yourself, everything you need is there and you can have a look at that. So now that I've been talking long enough, we will see on AWS that K-native has scaled me to zero and I'm no longer using those resources that are no longer needed. Okay, with that, I am going to end my presentation. I'll provide you with some contact information here, some information about the project and I will now open it up for questions and discussion. Thank you very much.