 Hello, everyone. My name is Reza and I'm a developer advocate at Tigera. Today, I'm going to talk about Kubernetes cluster mesh with open source Calico. I'm going to start with what is project Calico, then show you a demo and walk you through some of the stuff like BGP overview and cluster mesh. Alright, so what is project Calico? Project Calico is a pure layer three approach to virtual networking and security for highly scalable data centers. We offer Calico, a free and open source networking and network security solution for containers, virtual machines and native host based workloads. Calico supports multiple architectures and platforms such as x86 and arm64. So you can basically install it on any environment. This modular architecture make Calico a great choice for any environment and gives you the required tools to be in charge of your software defined networking traffic. As I mentioned, Calico has a modular architecture similar to Kubernetes. And it offers multiple choices that gives you the freedom to tune your cloud environment in a way that will be the most optimal for your scenario. Today Calico offers four pluggable data planes that can be switched on depending on your needs and environment. Standard data plane is based on IP tables that provides fast networking security and compatibility to all environments. Calico EVPF data plane is another data plane created by Tigera, which uses the power of EVPF programs to provide blazing fast networking and security for your environment. EVPF data plane offers capabilities such as complete coop proxy replacement, source IP preservation and DSR or direct service return. If you're running a hybrid environment, you can use Calico for Windows, which is based on Microsoft HNS and can deliver networking and security to your Windows nodes. WPP is the new US data plane for Calico. It was an external contribution from Cisco. Currently it is in its beta phase, which accelerates the networking experience by utilizing the power of user space programming. In fact, EVPF and HNS are some of the foundational technologies that provide networking, security, observability, image assurance and runtime threat defense in our enterprise solutions at Tigera. Our community has more than 9,000 Slack members and 320 contributors on GitHub. There are around 8 million nodes per day, which are covered by Calico, either people testing or they've been running Calico for six years. With that being said, let's get to the cluster mesh bar. Alright, I've got two clusters, two self-managed clusters that I created with Terraform. Don't worry, everything that I'm showing you will be available on GitHub, so you don't have to write down any codes or anything. It's just a QR code that I will share at the end and you can basically go through everything that I'm typing. So, and I've got a cube config file that has access to these clusters. Each cluster has a control plane node and a worker node. Alright, so first of all, throughout this video, I'm going to use dash dash context to switch between these two clusters. So cluster A is a cluster in AWS. It's on US West 2 and cluster B is another cluster on US East 2. Alright, so let's get started. So first of all, I'm going to tell Calico that I want to run BGP in my clusters. Now, in order to do that, I need to figure out an AS number or autonomous number. An autonomous number is more like an ID. It's a way for our cluster to represent itself as a router to other routers. I'm also going to write listen ports. This is going to be the port that BGP will listen on and form BGP connections on. There is node to node mesh, which I disabled. It is enabled by default and service cluster IPs. So this is very unique here because I do want my service IPs to be propagated when routing happens. We'll get to that in a second. Alright, so cluster A is going to be 6501 and cluster B is going to be 6502. Next, I need to create BGP peers. So this step is just to tell Calico that I want to use BGP. But in order to form BGP, I actually need to tell Calico that who's going to be my peer. For that, I can use another unique resource, which is called BGP peer, this is cluster A. So I'm basically telling Calico that use these IP addresses to talk to autonomous number 65002. And I'm going to do this same thing for cluster B. But this time I'm using 6501, which is the autonomous number of cluster A. Now that the BGP peers and BGP configuration is done, we can take a peek at what is happening. We can actually use a binary file inside the Calico node pod. So BERT CL is the binary that can talk to BERT who's doing the whole BGP peering and routing stuff inside the Calico node pod. Now if we look here, we do have two BGP peers, but both are stuck at the connect state. Well, this is because at the moment our two VPCs don't have any routes to each other. Keep in mind that I'm using two clusters in two different VPCs. So first of all, we need to figure out a way to establish a connection, either a VPN or a tunnel or any sort of thing. Now, since these two clusters are on Amazon, we could actually use something called VPC peering. So in order to do that, all I need to do is I need to get the ID of my VPC in the region USVS 2 and the ID of the other VPC in US East 2 and actually use a feature or use something called VPC peer connection. And that will make a request to enable a communication between these two VPCs. Now, since these two VPCs are using different IP citers, I also need to enable some sort of routing. So if there is any device in there that wants to talk to the other VPC, it needs to know where it should go to. So to verify, we could actually do context cluster A, get nodes, a wide, as you can see here, IP addresses are 172.16 to 107. And my cluster B adds 172.17 to 157. So these are slash 16. So the first two octets will never change. All right. So now that we got the cluster names and a VPC connection, let's go make a route. To create a route, all I need to do is figure out which routing table. So here, since I'm using tags, I can go ahead and say, all right, AWS, give me the route table that I have that is using this calico demo tag. And I can store this for cluster A and cluster B to environment variables. After it's done, I can actually go ahead and figure out what was the ID for my VPC peer connection. So here is the VPC peer connection ID. And store all of these in my terminal. So all I need to do right now is to just first, I need to accept the VPC peering connection. That means the previous step that we did with create VPC peering connection is just a request. Somebody needs to grant this request. So here, I'm accepting that request. Now the VPC peering is established. And now I can go ahead and use all the environment variables for route ID cluster A and peer ID to actually tell AWS that I want to be routing between these two VPCs. All right, now that this is done, let's go ahead and take a look at our BGP status again. Well, it's still trying to connect, but it can't. Well, one thing to remind you of is whenever you are using a cloud provider, there is a security group or some sort of rule that is preventing you from talking to other resources. By default, what each cloud provider would tell you is a cloud provider is secure and safe. And that is because in most cloud providers, there is an implicit deny for everything. I need to go ahead and say, all right, I need to enable this flow or I need to enable this communication to happen. So with that in mind, and if you remember, we're going to open the port 179, which at the start of this video, we take a look at it in our BGP connection configuration, and it was for BGP. All right, so here, I'm going to grab the security group that my cluster A is using. I'm going to store that ID into an environment variable. And then I'm going to authorize that security group to be able to communicate on port 179, if cluster B or cluster B cider is talking to it. So the cider is for VPC on the other zone. And I'm going to do the same thing for the other cluster done. So right now, if we take another look at our BGP, all the stuff are established. We can also use show route to see which routes are being propagated. So it's a good time for us to hit another problem. So let's see. First, I'm going to create an nginx workload on my cluster A. And I'm going to use service for it, a node port service. Next, I'm going to use a couple of commands to get that IP address to these services and pots. And run a net shoot pod to actually try to access this pod. All right. So first of all, let's do a curl. It's not happening. And let's do a curl for the service. Now both are not happening. Well, why? If you look, there is a route. So my pod knows where to go. But we need to know this. At the moment, every routing decision happens by our default gateway, which is Amazon. Now, Amazon doesn't know about the stuff that I have in my cluster. So these IP addresses, 1053, 105321, these are not the IP addresses that are inside Amazon Routing Table. Now, a neat trick that we can do with Calico is to tell each cluster that what are the IP addresses that are in the other cluster. And tell it to use encapsulation if it wants to talk to those IP addresses. So in order to do that, I'm going to create two IP pools for each cluster. And one is going to be services that are in the other cluster. And the other one is going to be the pod sider. Now, there are a couple of things that are very important. One is disabled. Now, disabled means I don't want Calico to give its local pods in cluster A any of these IPs. So that's why it's disabled. I also want IP IP mode, which is the encapsulation to be used whenever a traffic that wants to go through that IP address is crossing a subnet boundary. That means if, let's say, 105201 wants to talk to 105300, then it will use encapsulation because these two IPs are in different broadcast domains. So I'm going to go ahead and do the same thing for the cluster B part of the... All right. I'm going to do the same for cluster B here. All right. Now that we got the IP pools in place, let's go ahead and actually tell Amazon that we are using IP pool. Again, I'm going to create another security group that allows IP IP traffic between these VPCs. Next, I'm going to tell Calico that an external entity will send IP IP traffic to it. So be prepared to receive it. And I'm going to do it on both clusters. All right. Now that everything is configured, let's go ahead and run some tests. So I do need the IP addresses for the nginx pod. And I'm going to use my cluster B to actually make a workload and parallel this. All right. So at the moment, I'm in cluster B. I'm on another subnet and I can access the nginx pod. Let's try the nginx service. Now, this is not going to work. Well, because in Kubernetes, a service has a couple of key points or a couple of configurations that we need to be aware of. One is external traffic policy. So by default, external traffic policy is set to cluster, which is going to be problematic for our scenario. And we need to change that to local. Now, if we, again, go ahead, use the same IP address, it should give us some results. And with that, we have communication between these two clusters. That was it. I hope you enjoyed the session. And thank you for viewing.