 Hello, and welcome to Inside Kubernetes Ingress, a KubeCon and CloudNativeCon North America 2020 presentation. I am Dominic Torno, principal engineer at Cisco. I focus on systems modeling, specifically conceptual and formal modeling, to support the development and the documentation of complex software systems. This presentation focuses on the concepts behind Ingress for Kubernetes. It does not focus on its possible implementations or on its possible set of features. Kubernetes Ingress is related to Kubernetes services. To deep dive into Kubernetes services, visit Inside Kubernetes Services, a KubeCon and CloudNativeCon North America 2019 presentation. What problem does Ingress for Kubernetes address? Ingress for Kubernetes enables the external consumption of a set of Kubernetes HTTP services hosted on one cluster via one HTTP endpoint. How does Ingress for Kubernetes address this problem? To enable the external consumption of a set of Kubernetes HTTP services hosted on one cluster via one HTTP endpoint, Ingress for Kubernetes addresses two different concerns, Network Ingress as well as Kubernetes Ingress. Network Ingress addresses the question of how to admit traffic into the cluster. Kubernetes Ingress addresses the question of how to route traffic within the cluster. A Kubernetes cluster is typically defined as a set of Kubernetes nodes, a set of physical or virtual machines. However, this presentation is not concerned with nodes. So we will reason about a cluster as a set of pods that ran, run, or will run on the cluster's nodes. The first topic of this presentation will discuss Network Ingress, the admission of traffic. However, as Kubernetes does not specify how to implement Network Ingress, leaving the implementation up to the operator of a Kubernetes cluster, we will discuss only the what, not the how. The second topic of this presentation will discuss Kubernetes Ingress, the routing of traffic. We will discuss both the what and the how. Before we develop a definition of Ingress for Kubernetes, we will spend the next few minutes to develop an intuition of Ingress for Kubernetes. In order to develop an intuition of Ingress for Kubernetes, we will develop an intuition of both Network Ingress and Kubernetes Ingress. First up, Network Ingress. The admission of traffic. Let there be two communicating end points, a service consumer and a service provider. The service consumer is not hosted on the Kubernetes cluster. It is external. The service provider is hosted on the Kubernetes cluster. It is internal. Network Ingress denotes a point or means of admission. Furthermore, Network Ingress implies directionality, crossing from external to internal. Next up, Kubernetes Ingress, the routing of traffic. Previously, there were two communicating end points, a service consumer and a service provider. The service consumer has to learn the address of the service provider to actually consume the provided service. However, a persistent trend complicates this picture. One monolithic service provider is broken up into many service providers, microservices. Now the service consumer has to learn the address of each service provider to consume the services. Kubernetes Ingress is a proxy, an API gateway that exposes multiple service providers as a single endpoint, therefore greatly simplifying consuming the services. Putting both together. Ingress for Kubernetes is a composition of Network Ingress and Kubernetes Ingress, where Network Ingress is the admission of traffic into the Kubernetes cluster, and Kubernetes Ingress is the routing of traffic within the Kubernetes cluster. In effect, Kubernetes Ingress is an API gateway. With an intuition of Ingress for Kubernetes, we will spend the rest of the presentation to develop a set of related definitions of Ingress for Kubernetes. In order to develop definitions for Ingress for Kubernetes, we will once again develop definitions for both Network Ingress and Kubernetes Ingress. So first up, Network Ingress, the admission of traffic. In software engineering, the distributed system is an unbounded set of components from here on out called endpoints. Endpoints communicate by exchanging messages via a network. The behavior of a distributed system is attributed to the behavior of its endpoints and the communication between them. The complexity of a distributed system is attributed to the autonomy of its endpoints and the intricacy of the communication between them. Without loss of generality, let's focus this discussion on two endpoints, E1 and E2. An endpoint is connected to the network via a channel. The network maintains an association between endpoints and addresses. From here on out, we will graphically represent this association as if the address is a property of the channel. We keep track of the sequence of send events and receive events in an endpoint's history. If an endpoint wants to send a message, it will place that message in its channel. An endpoint placing a message in its channel is represented by a send event. The network picks up the message from the sending endpoint's channel and determines the receiving endpoint's channel and places the message in that channel. The network placing a message in an endpoint's channel is represented by a receive event. In this network model, send events are tagged with a target address. The network places the message in the channel of the endpoint whose address matches the message's target address. This can also be represented graphically as a time-space diagram. Each timeline represents an endpoint's history. Empty circles represent send events. Filled circles represent receive events. A pair or tuple of corresponding send and receive events is called a flow. So far, we have applied a global point of view. In this model, we are able to take the view point of the all-knowing observer. We can observe both the channels of E1 and E2 at the same time. Conversely, E1 or E2 cannot. E1 can only observe its own channel and in our model its own address. It simply cannot reach beyond. The same is true for E2. E2 can only observe its own channel and its own address. So in order for E1 to send a message to E2, E1 first has to learn the address of E2. The same is true for E2. In order for E2 to send a message to E1, E2 first has to learn the address of E1, a process called endpoint discovery. Moving towards the Kubernetes network model. In Kubernetes, network addressable endpoints are pods. The Kubernetes network model specifies that any pod can communicate with all pods without network address translation. The Kubernetes network model does not specify whether external endpoints can or cannot communicate with pods. As a consequence, depending on your cluster, network ingress may be trivial or complex to implement. As we discussed earlier, we separate the set of endpoints into external endpoints and internal endpoints who communicate across that line of separation. Here, we consider endpoints 1 through 4 as being external endpoints and 5 through 8 as being internal endpoints. In effect, pods, given the separation of endpoints into external and internal endpoints, we can classify the communication between endpoints according to the membership of the source and target of the communication. There are four possible combinations. In the first combination, source is a member of the set of external endpoints and target is a member of the set of external endpoints. This particular type of flow does not have a name. In the second combination, source is a member of the set of external endpoints and target is a member of the set of internal endpoints. This particular type of flow is called north-south traffic. In addition, given the directionality, this combination constitutes network ingress. In the third combination, source is a member of the set of internal endpoints and target is a member of the set of external endpoints. This particular type of flow is again called north-south traffic. In addition, given the directionality, this combination constitutes network egress. In the fourth and last combination, source is a member of the set of internal endpoints and target is a member of the set of internal endpoints. This particular type of flow is called west-east traffic. In conclusion, network ingress can be defined as a set of all flows that originate outside the cluster and terminate inside the cluster. Next up, Kubernetes ingress, the routing of traffic. Kubernetes defines a Kubernetes ingress object. In effect, a Kubernetes ingress object defines a collection of HTTP request-level routing rules that determine the target of that request. Ingress matches an HTTP request's path and host header against its routing rules to determine the target Kubernetes service to proxy the request to. This example illustrates a Kubernetes ingress object. In effect, this ingress object defines a collection of four request-level routing rules. In my personal opinion, these are best represented as a decision table. For example, the first rule matches an HTTP request with a host header of foo.org and a path of slash a to proxy to a part that matches a service named foo.a on port 8080. Again, represented as a row in the decision table. The third rule matches an HTTP request with a host header of bar.org and a path of slash a to proxy to a part that matches a service named bar.a on port 9090. And again, represented as a row in the decision table. Represented as a time-space diagram, when the Kubernetes ingress proxy receives a request, it matches the request against the decision table and forwards the request so that a part that matches the target service receives the request. Why do I say forwards the request so that a part that matches the target service receives the request and not simply forwards the request to the target service? Because there are implementations that implement their own part discovery in accordance with Kubernetes services, but do not rely on the discovery implemented by Kubernetes and Kubernetes services. Next up, the Kubernetes ingress controller, the control plane component. Kubernetes sent us around the notion of Kubernetes controllers and Kubernetes objects. Kubernetes controllers continuously read and write Kubernetes objects. Core controllers interact exclusively with the API server to read and write a set of Kubernetes objects. Edge controller interact with the API server to read and write a set of Kubernetes objects, but additionally communicate with other components in the data plane. Let's examine a few familiar examples. The Kubernetes replica set controller is a core controller. It interacts exclusively with the API server. The replica set controller reads replica set objects and writes port objects. The kubelet is an edge controller. It interacts with the API server and with the container runtime. The kubelet reads port objects and instructs the container runtime to execute containers accordingly. Similarly, the Kubernetes endpoints controller is a core controller. It interacts exclusively with the API server. The endpoints controller reads service objects and writes endpoints objects. The kubet proxy is an edge controller. It interacts with the API server and with the Linux netfilter module. The kubet proxy reads endpoint objects and instructs the netfilter module to create network address translation rules so that a message sent to a service IP address will be forwarded to a port IP address with the port being a member of the endpoints. Now on to the ingress controller. An ingress controller is an edge controller. It interacts with the API server and with an ingress proxy. The ingress controller reads ingress objects and instructs the ingress proxy to create routing rules according to the decision table specified in the ingress object. Lastly, next up, the Kubernetes ingress proxy, the data plane component. As discussed earlier, network ingress may happen before or after Kubernetes ingress. So there are two possibilities. The ingress proxy may be an external endpoint or the ingress proxy may be an internal endpoint, apart. But either way, the task of the ingress proxy is to accept the request, match the request against the decision table specified by the ingress object and installed by the ingress controller and forward the request so that a part that matches the target service receives the request. So in conclusion, Kubernetes ingress can be defined as a set of all flow pairs so that the first flow terminates at the proxy, the second flow terminates at a part and there exists a rule in the decision table so that the request of the first flow matches the conditions of the rule and the part matches the target service of the rule. Truly, not as straightforward as the first formula. Let's conclude. Ingress for Kubernetes encompasses two aspects. Network ingress, the admission of traffic into the cluster and Kubernetes ingress, the routing of traffic within the cluster. Kubernetes ingress is composed of three building blocks. The Kubernetes ingress resource, the Kubernetes ingress controller and the Kubernetes ingress proxy. However, Kubernetes provides only the ingress object, ingress controller and ingress proxy are third-party components. In effect, the Kubernetes ingress object defines a collection of HTTP request-level routing rules that determine the target of that request. The ingress controller reads ingress objects and instructs the ingress proxy to create routing rules according to the decision table specified in the ingress object. The ingress proxy accepts the request, matches the request against the decision table specified by the ingress object and installed by the ingress controller and forward the request so that a part that matches the target service receives the request. And finally, what is the difference between Kubernetes ingress and an API gateway like the ambassador API gateway? That, of course, is a trick question. In effect, the concept of Kubernetes ingress is the concept of an API gateway. And in effect, the Kubernetes ingress object is a standardized configuration for API gateways. Popular API gateways, like the ambassador API gateway, can be installed to read the ingress object and act as the ingress controller and the ingress proxy. If you are watching this presentation during the conference, I will be happy to answer your questions online. If you are watching this presentation after the conference, I will be happy to answer your questions via email. But either way, thank you for watching Inside Kubernetes Ingress.