 Hello. Hello. Good afternoon. Thank you for coming to the talk. My name is Ramiro, I work for Pivotal San Francisco, and I focus primarily on networking. The reason why we wanted to have this talk in particular is because the pattern of service meshes is a pattern that has been starting to percolate in the industry for the last two years approximately and some communities are probably more aware of that pattern than others. We have noticed in different conversations with customers specifically in Pivotal that the CF community in general is still not very aware of the pattern of a service mesh. So we wanted to do like a very, very basic with no assumptions whatsoever, and there are many service mesh types, right? And we're going to focus primarily on Istio, which is the one that we're interested in, and I want to cover why we're interested in that particular one. And the idea obviously is to keep it hopefully interactive. I'm going to try to leave a few minutes at the end. This is a conceptual talk. It's a basic talk with the whole purpose that you guys walk out of here understanding the pattern, understanding why this pattern matters, and what's going to look like for the future of the industry. And this is as big as that, the future of the industry. It's all about microservices, and this is not an understatement. Microservices in general, they have changed the way networking should work or networking should react. It's not just a matter of speed. As you know, microservices can be created and destroyed in a matter of microseconds or milliseconds in many cases, right? Not all workloads are like that, of course, but it potentially can happen. And when that happens, what's going to be the impact on the network, right? Primarily, what's going to be the impact when you have, instead of hundreds of VMs, which you used to have before, now you have hundreds of thousands of microservices, right, that all need to have their own networking characteristics. So the microservice expectations of the networks are pretty high. Microservice have requirements, like every other type of application in history has had requirements. Specifically, it requires the network to exist, to be available. It relies on the network. This is what we do in computer science. We layer things. We've been layering things since the beginning of computer science. So a microservice as an entity, as a software entity, requires a network to be there, to be available. It requires to be fast. Microservice doesn't have to concern itself about the speed of the network. It has to make an assumption that the network is essentially fast. It has to make the assumptions that the network provides some degree of security. Let's say firewalling and functions like that typically happens outside the purview of the microservice. There is obviously some security that happens within the microservice, right? Typically, single sign-on capabilities and so on are managed at the microservice level. But there is a basic expectation that the network will provide some degree of security, at least periminal security. And there's also an expectation in these days that the network is going to provide a relatively low latency. And this is important, specifically as we move on to applications that use heavily other microservice to provide a single output. So you have the expectations that they all behave like a consistent system across the board. You may have new applications that are going to be consuming hundreds or thousands of microservices, and you want to make sure there is consistency among them. So not one single microservice is becoming the bottleneck for others. So there's a lot of those patterns that start to matter. And there's also the expectation that the network is going to be transparent. You don't care in this modern world if your network is based on AWS or is based on whatever underlying SDN is provided, either by you or your data center or anybody else for that matter, you just care that it's available, it's there, it's somehow available to you. And of course, we would love to believe that networks should never fail, right? Which we all probably know better, right? So there is some degree of certainty that things will fail. How does a microservice needs to react in terms of a failure? That's another aspect that needs to be considered within the development and deployment of these microservices. The network on the other hand, the network also has some expectations for the microservice, right? It's not like the network knew ahead of time of the microservice, right? This is networks predate, typically the generation of microservices. However, networks somehow expect that the microservice will implement its own service discovery. The network function, as it is, does not provide you generally with any sort of service discovery. There are other mechanisms for service discovery that are implemented as a network function, but typically it's something that needs to be managed at the application level. The platforms typically are very good in that, right? Software systems like Consul, for example, or Eureka are very good doing that functionality, but it's not necessarily a functionality that comes from the network. So take a look at what I'm trying to say here. I'm trying to say that all this is from the perspective of the network, right? So the network really doesn't necessarily implement its own service discovery. It expects the application or the microservice to implement its own app-level flow control. The network provides you with some degree of flow control, right, which is typically based on the TCP layer, right? TCP has its own mechanism for the window size and so on, adjustments for the flow control, but that's a very low-level facility. You don't expect or you wouldn't want that modern developers have to concern themselves with that. In the ideal world, I should be abstracted from them. They shouldn't have to think about that. At the same time, you need to implement your own network security. Obviously, the network security externally to the microservice is provided, obviously, by the network, like firewalls or IDS and IP assistance and maybe outside the purview of the microservice, but there is an expectation that the microservice itself is going to have to provide some degree of security. Implement its own protection in case of a network partitioning. This is a very important element in distributed systems. Network partitionings happen. That means that if you have multiple instances of a service, let's say HCD is a good example, right? Let's say you have three instances of HCD and then as a partition occurs, HCD has intelligence within the system to understand what happens during that partition. If the partition is over a certain amount of milliseconds or seconds, then suddenly you now have two clusters, one cluster of two, one cluster of one. So, but there is obviously that's an undesirable effect, but it is something that the application had to learn to deal with because the network partition is something that will occur eventually in any network whatsoever. So, protocols now have started to emerge. Once you start mixing these two worlds, these two paradigms, you start seeing that microservices and networks, they start creating these new patterns, these new protocols like HTTP2 or like GRPC, fantastic protocol that is based on HTTP2. For IoT or Internet of Things, you start seeing the emergence of things like MQTT or co-app, for example, newer protocols that didn't exist years ago. However, this is still not sufficient. There's still need for things like HTTP, 1.1, there's still need for encryption everywhere, there's still needs for sockets like raw TCP sockets or even raw UDP sockets, right? There's the need for multi ports, applications that have specific needs for multi ports. Some other ones are dynamic ports, which makes it even more difficult, especially the voice over IP and media applications, right? So, unfortunately, nothing of this is really enough so new patterns needed to emerge to satisfy the demanding needs of these microservices. Now, at the same time, microservices have also complex needs that are not easily fulfilled. Service health checks, for example. The network doesn't provide any mechanism natively to provide you with those health checks, right? Extensive metrics. Well, the microservice will generate those metrics and they need to go somewhere, but the network is technically not responsible for any of that stuff. You need to send them somewhere, right? You need efficient patterns like circuit breakers that have become very popular in the spring boot world and so on. You need higher level flow controls, like Managed QoS, that you want to make sure that some microservices are not overwhelmed by the pressure of others. And you need especially advanced policy controls. You want to make sure that not only the developer is setting the policies of who can talk to whom, but also external entities, say the InfoSec teams. The InfoSec team may want to have horizontal adjacent policies that are superseding the policies that developers are creating for their own applications, right? And, of course, the networks can't really provide all that themselves. Now, 30 years ago, when I started doing networking, we would have created a protocol for this, right? Back in the day, we used to do like, okay, there's no protocol that could do that. We'll just do another one. IPX, bunion binds. We did all sort of different protocols for different purposes because it was okay, right? You can create a new protocol. We would have done that today if we could, but unfortunately we can't. IP won the battle. IP is everywhere. The fact is that now, we need to start augmenting the network. We need to assume that the network is what it is. There are some intelligent protocols, very interesting things that are coming out of the pipe, like Quick, for example, which is a protocol that Google is actually experimenting quite a bit of. There's BBR for congestion control. But these are essentially efficiency models over the same IP paradigm. Unfortunately, you still need to have a more complex layer on top of that network. So what we have done is that we have decided to replace, instead of having this direct communication layer where you have all these microservices talking to each other directly, natively, right? We decided to change the model and overlay a new system in between. In this case, the system, we call it sidecars, right? The sidecars are essentially very small and very efficient, highly sophisticated proxy systems that sits in between the microservice itself and the network. So the microservice does not really talk to the network. However, the microservice doesn't have any awareness of them. The microservice believes it's talking directly to the network, but in reality, it's talking to a proxy. This behaves very similar to the way you normally work with a proxy today. There's almost no difference, except that this proxy, as you will see in a moment, they do a lot more than the proxies that you're probably used to. But essentially, conceptually, the beauty of this is that now the communication, all the aspect of the communication, without having to create a new protocol, we now are creating a new layer on top of the pre-existing protocols. The advantage of this model is that now you have an entirely new data plane, and this data plane is leveraging things like mutual TLS. So every little blue line that you see here, for example, is a mutually authenticated TLS communication line. Done directly in layer 7. So all the intelligence is being moved upwards in the stream. No longer, you need to rely on things like, for example, IPsec encryption, which is fantastic. I love IPsec. I've done tons of projects with them, and I'm a big fan. However, not everybody allows that. And that's the job for my friends at Microsoft there, that do not allow native protocols 50 and 51 in their network, which is a standard IPsec things. And there's many other networks where you simply are not going to allow things like TCP or UDP or even ICMP for that matter. However, mutual TLS works everywhere. Now, obviously, these sidecars are a data plane. They still need a control plane. This control plane is the one that basically adds the intelligence layer. How are you going to tell all these little sidecars to do collective magic? That's the whole point of this. And that is essentially a service mesh. And I'm done. Bye. Not really. But no, what I'm saying is, essentially, a service mesh is as simple as that and as complex as that. Right? There's a lot of intelligence that goes behind scenes. Now, the market today, if you just look at the market as it is, you have, in the data plane world, you have things like LinkerD, Anvoy, HAProxy, Traffic, and there's other ones that keep coming up. They manage those connections really well. They have a lot of features in those connections. They're all very rich in terms of capability. These are tiny little proxies. They're all efficient. And they do one thing only, and they do it really well. Some of them have more functionality than others. Some of them have more contributors than others. But all of them are really good solutions. And in terms of control planes, you have other ones that are starting to pop up in the market. You have Istio. You have Synapse from Airbnb. You have Nelson. And you have other ones also popping up in the open source community. Nelson is more Git-centric, for example. So you have different sort of systems out there that are trying to position themselves within this. Now Istio, in particular, is a project that was originally sponsored out of IBM, Google, and Lyft, right? And we say Lyft in particular, because they are the ones behind the Anvoy proxy, which is the one that Istio uses. At Pivotal, we're particularly interested in Istio, primarily because it's first and foremost platform agnostic. So it doesn't have a requirement to be unique for Kubernetes or unique for Cloud Foundry or for any other system. It can be adapted to any sort of platform. It's already been progressively added into Cloud Foundry. There was a talk this morning about that. There's more talks coming. I have a schedule of the talks. We're going to be talking more about how Istio is getting slowly but surely inside Cloud Foundry. The very stage that you're going to see about this is the inclusion of Anvoy proxies being deployed with every app. Typically, in a Cloud Foundry world, when you say CF push, you push an app, and an app is a single type of container entity. It doesn't matter how many you have. You can have any number of instances, but essentially it's one single type of binary. Well, that's about to change with this. Now, in addition to adding that single type of binary, now you're going to have a sidecar alongside with it. And that sidecar is going to be an Anvoy, and that's the one that is going to be controlling the communication. The beauty of the implementation on Cloud Foundry is that we don't lose any of the simplicity that you're used to. Cloud Foundry became popular with the developer community because it's simple. You do a CF push and magic happens. Nothing of that is going to change. What you're going to do, you're going to inherit these advantages that you get with a service mesh. Now, obviously, in other Clouds or other orchestrators like Kubernetes, for example, you don't have those constructs. So they're much more manual. You need to manually do things, manually enable the service mesh, manually set it up. That's some of the things that we're going to be working with the community to, over time, improve in this process. As I was mentioning, it's backed by Lyft, by IBM and Google. All partner companies, we partner in different areas of the business, is very, very, very lightweight, specifically the Anvoy system. It's written in C++. It's extremely powerful, extremely sophisticated, it has a lot of different features, very robust API, and it's so small, with so much functionality, that it's literally a no-brainer for us to just use it as a standard base of the system and implement it alongside pretty much anything else that we do. It has a very dynamic and vibrant community. It's a young community. It's willing to do changes dynamically, depending on what's needed. And they are heavily centered in terms of plugability and extensibility of their product. They have capabilities as distributed tracing, for example. Zipkin is a natural element of distributed tracing, and it's standard with Istio, it's standard with Anvoy in particular. Anvoy also generates a ridiculous amount of metrics. It has so many metrics that can overwhelm a normal system. So that is very, very powerful. In some cases, we have to decrease the amount of metrics that we generate out of Anvoy, because it has so many, which is fantastic. And finally, it has distributed security. Istio as a whole has the capability of maintaining this end-to-end TLS communication among all the endpoints and also have the capability of creating and recreating and managing the TLS keys that are typical in this type of environments. Anvoy in particular, I was mentioning before, is a C++-based system, layer 4, layer 7 proxy. Very low memory. It's been battle tested already in places like Lyft. This is not news. Being out there for a while, it has over 100 plus services being deployed over 10,000 VMs and two millions of requests per second. This is a system that actually works. It works very well. There is, of course, there was a version 1 API, now there's a new version 2 API that keeps growing and is more thought out for distributed systems as a whole. It has native HTTP 2 capabilities and GRPC capabilities out of the box. This is nothing that you have to add or plug in or anything like that. Is zone-aware load balancing with failover? Has the capability of understanding AZs as well? Has the capability of doing health checks, circuit breakers, timeouts and retries. It's extremely sophisticated as a piece of software and it's also completely transparent to the network. Overall, it has so many features that I would highly recommend that you guys look at their web page and look at the entire list of features that they provide. All of those features are somehow exposed in the Istio environment. So whenever you have an Istio environment, you're going to be able to use those features directly of Envoy. Envoy is basically producing the data plane that you need being controlled by the Istio control plane. The other interesting corollary of this is that there is now a big change in the discussion around what is east-west traffic versus what is north-south traffic. In Cloud Foundry, that's a very common conversation. In fact, sometimes it's a heated discussion. North-south traffic is pretty much one of the things that you do in order to access applications in Cloud Foundry. You come via the load balancer through the go router and then you go into the application. And when the microservice needs to talk to another microservice, if you're not using CFN working, which is rather new, you have to go all the way around the load balancer. So you generate a lot of north-south traffic as opposed to the modern data center designs that are based on spine leaf architectures where typically you optimize for east-west traffic. Well, in this model, now that you have sidecars everywhere, north-south-east-west become literally irrelevant. Everything is part of the same mesh. The traffic will flow directly from the north. There's a single entry point. But now when microservices need to talk to other microservices, the mesh literally is going to make the decision how to reach the next microservice that you need to talk to or how to reach an external service that you need to talk to. They're all part of the same mesh now, optimized for that purpose. So this is pretty game-changing, if you guys think about it, especially if you've been involved in the Cloud Foundry community for a while. This not only has the potential to completely replace the routing tier in Cloud Foundry, but it also has severe implications also for Kubernetes, how Kubernetes is going to evolve and transform itself as a product in a more usable fashion. This is going to give Kubernetes new usability levels. If you want to dig down deeper into how Istio itself is built, Istio has different components as part of the control plane, it has three major components that we're going to talk to. Primarily, pilot, mixer, and Istio off. These are the three different pieces. One of the things that this team did very, very well is plugability at every layer. Each one of those layers is highly pluggable. They all have good APIs. They all have the capabilities to be inserted into pre-existing models. And there are no dependencies at the moment with any other orchestrator system out there, which is an absolutely fantastic concept. Mixer. Mixer is the center point for the policy evaluation and telemetry of the entire system. The policy system is pluggable here. Think about this, the big concept here behind this. If you create your own policy language, because you want to create a domain-specific type situation, you can create your own policy language and you can plug it into Mixer, as long as you make sure that all the constructs match and you use the Mixer API and you plug into that system. One example, for example, we're looking seriously at the guys of OPA, for example, OpenPolicyAgent.org. OpenPolicyAgent, for example, they have a multi-layered domain-specific language, very sophisticated language, right? They apply different sort of policy mechanisms to different layers of the infrastructure. Well, that system is now pluggable with Mixer. That means that if I choose to use that system, I can just grab it, plug it into Mixer, and now I can use my OPA language in order to define and describe the policies of my entire mesh. The potential for this is fantastic. It's literally infinite potential possibilities of slicing and dicing any sort of platform that you want. And again, this is irrelevant if you're in Cloud Foundry, you're in Misos, you're in Kubernetes, you're in any other thing that comes out in the future. It supports Mixer supports precondition checking, quotas and rate limiting. It has a primary point for extensibility, as we mentioned before, and it has specifically operator configuration model, right? So you can focus on the operator side of things. Obviously, in a DevOps environment, where, say, a developer can actually own an entire Kubernetes environment, like what we'll see with PKS and Kubo, in that sense, it makes a lot more sense for them to control their own destiny. They can plug, let's say, per every Kubernetes class, and they can plug their own components within Mixer. It doesn't have to be to have any sort of relationship with other clusters that are being provisioned in parallel. More about that later. Pilot. Pilot is the system that actually plugs into the entire control plane to maintain the lifecycle of the Anbo instances. And it has a very good API for service discovery. Now, you need to have a service discovery system that can plug into this. That's one of the things that we're doing, for example, in Cloud Foundry right now. In Cloud Foundry, the new service discovery system that is being created, is being programmed as we speak, that system, one of the requirements that it has is to be plug-able with Pilot. So in the future, when Pilot is fully integrated into Cloud Foundry, you're going to have the capability of using the service discovery that we produce in Cloud Foundry, which is actually based on DNS internal names, as well as Bosch DNS, which is being used as well as underneath the system. There is obviously load balancing pools and routing tables. When I say routing here, we're talking layer 7 routing. We're not talking about the typical layer 3 routing that we're used to. This is layer 7 routing. This is application A talks to application B and this is the way to reach application B and this instance of an application B. If I want to have blue-green deployments, I can also do that. That is a routing function for the mesh that will determine when an application is going to start fading from point A to point B as part of a standard mechanism for the service mesh. East to off at the same time, another very powerful construct. It gives you strong service to end-to-end service for end-user authentication and mutual TLS. It has built-in identity and credential management and in the future, it will have role-based access control and authorization hooks. They're actually in the roadmap. So this is very fast. I know because this is a short talk. We have half an hour only. So what about Kubo now? Well, Kubo, as you know, and using the expression that my friend Colin loves to talk about is invented to delight, as he likes to say, a person who creates Kubernetes clusters. So Kubo, per se, or whatever it's going to be called now, I can't keep up anymore. Kubo is essentially a product that is not a Kubernetes distribution by itself. It's a software tool to create Kubernetes clusters. So other people or yourself can consume those clusters. If you're a developer that wants to consume a cluster, you can just go to your service broker and say, hey, create me a Kubernetes cluster, create me a Kubernetes environment, and I can use that. It's as vanilla Kubernetes as it's possibly capable of doing. This is one of the goals, is to keep it as up-to-date as possible. And of course, it's all controlled by Manage, by Bosch, which is one of the advantages of this. You can upgrade the entire control plane of the system without necessarily touching the workers. You can upgrade the workers later if you choose to do so, but the control plane can also be upgraded directly via Bosch. It has strong isolation and multi-tenancy, which is achieved by Bosch primarily. You don't have to worry about the OS anymore or the capabilities. All that is done by the stem cells. And the other very important aspect of this is that Istio, like Kubernetes, it does not do multi-tenancy very well. In fact, at the moment, Istio doesn't do any sort of multi-tenancy. So that is a perfect fit for us. This is really what we're telling Istio on Kubernetes plus Kubo is the goodness here, because now Kubo primarily provides you this isolated clusters, and the isolation is guaranteed by Bosch. This is hard isolation guaranteed by Bosch. Now you can have your Istio environments directly on top of that. Each developer that produces a cluster can choose to install their own Istio environment automatically, right? Or manually, if you want to do that too. And now consume Istio within your service mesh, within your internal cluster. This is the present. This can be done today, right? In order to set it up, super simple. Just literally all the steps that you're going to document later, you can see them later in the recording or just go to the website. The installation is extremely simple. You just need to deploy your Kubernetes environment, assuming that you have the prep done for your own cloud, right? Then set your cloud, your Q config directly to point to your cluster that you're trying to do. Then get the Istio release, and as part of the process of getting the Istio release, it will get automatically deployed. And if you choose to do the empty LS for all the sidecars with automatic injection, it all automatically gets deployed with this Istio initializer. When you install Istio initializer, that means that every new app that you push into the Kubernetes cluster that you created is going to automatically get the sidecars inserted. And you can then control the mesh in any way that you choose to do so. So then you have Verify Istio that is installed and running. This will do it with these commands, right? And then finally, deploy your app. And this is the way you deploy your app. You have two mechanisms to deploy it. I'm not going to spend time on that because I just don't have much time. Now, the future is great, right? The future is great. So one possible future, you know? From your point of view, I don't know tech stuff, right? That's a line from Terminator for you, all-timers. So I want you to think about these possibilities, right? This is an existent, not vetted, but I just want to make sure that you guys get a sense of what is possible here. Say it's Kubo create me a Kubernetes cluster with Istio pre-installed in my local VCR environment. Today, we cannot do that, right? Kubo create vanilla clusters. But nothing will stop us to evolve Kubo to do this, as well, to create pre-ready Istio environments, right, with Kubernetes there. Furthermore, I say now use the open service broker or demand service broker to create my Bosch Manage Postgres cluster fronted by an Anvoy proxy for my own use and make it part of my mesh. That can also be done. As of version 0.2 of Istio, you can now support VMs as members of the mesh, not only containers, right? Furthermore, and while you're at it, extend my mesh to org foo and spaces bar and bas in my development Cloud Foundry or my Google Cloud using my credentials. So imagine what we're talking about here, merging all the system, creating a service mesh that will go from Cloud Foundry all the way to multiple Kubernetes clusters for the developer's enjoyment and delight. Again, following my friend Colin's terminology here. And now even create me a new Kubernetes cluster on my AWS environment and furthermore, extend the Istio mesh there. All those things are actually feasible to do. How many of these are we going to do? When are we going to do that? It's always going to be dependent on you, especially if you are a customer, right? You are the ones that need to tell us, yeah, this resonates with me. This sounds like something I want to do, right? And if that's the case, I want to hear from you, literally, because actually we're in the process right now building roadmap for this kind of thing. And we're going to work in conjunction with the community. This is not an insular thing. This is all community-based. This is all open source code, right? We're going to be working with everybody on this area. But need to adapt. Need to adapt. SDM vendors will have to up their game because now all the intelligence is moving up. To layer seven. All this beautiful micro-segmentation, all those goodies that they used to sell us so hardly. Now, suddenly, I wouldn't say they don't matter. They do matter for one purpose because they need to protect the mesh, right? They need to protect it from denial of service dax and so on. But essentially, that's the role now. So they're going to have to figure out a way to insert themselves into a world where the intelligence is moving upwards. Orchestrators. And this is also valid for the Cloud Foundry community and for the Kubernetes community. We're also going to have to up their game. Because essentially, once you have a mesh that can expand in different directions where essentially all the components are going to look pretty much alike, then you better have your game. You've got to figure out a way how to keep adding that secret sauce that people will want to buy your product or use your product if you're using all the open source. Unvo is great, but, for example, it doesn't have the capability for more specialized things. Like, in the NFV world or network function virtualization for telecom, for example, we love to use SCTP protocol, which is not there in the list. Unvo doesn't support that. So should we create an alternative to Unvo? Should Unvo grow in that direction to also support SCTP? What about native MQTT or co-app for IoT protocols? So all those things actually matter, right? Which one of them are we going to be able to support or not? It's going to be over time. We're going to have to figure out. Sense of realism is too young. It's new. It's a very new protocol. There's not a lot of best practices for HA right now. There is no federation capability. A lot of the things that we're talking about, depending on the capability of federation, it doesn't support that yet, right? They're obviously thinking about that. There needs to be integration and cooperation with the SDN layers. There's none of that yet, right? All that needs to happen. The Tigera guys are probably the only ones that are more advanced thinking about that area. With Calico. No options about the end-to-end policy to provide flexibility. It can be create. It can be complex, et cetera, down the road. So essentially, there's a lot of still unknowns in this area. There's a lot of things that are being developed. The big message for you guys is get educated on this. And if you have ideas, let us know. If you have some things that we haven't thought about, let us know. Many of these things are going to be incorporated in future roadmaps and it will be up to you to determine which ones. And unfortunately, I'm out of time. So thank you.