 Cool, welcome to TubeCon 2020, North America edition. My name is Mayan Kumar. I'm a software architect at Salesforce and with me I have my colleague Hayek. I'm responsible for our public cloud Kubernetes strategy and helping Salesforce migrate our services to public cloud in cloud native way. Thanks, Mayank. Hi folks, I'm Hayek. I'm a senior virtual member of technical staff here in Salesforce. I work on projects that cover areas such as security and distributed systems. Here in Salesforce we are big believers in open source and we try to contribute as much as possible. And so as part of those contributions earlier this year we open sourced a generic solution for sidecar injection for Kubernetes board codes. So let's dive in. Cool. So when Kubernetes announced the support for commutating admission controllers in 1.9, the sidecar pattern really became a first gas engine of Kubernetes. Many of the infrastructure teams at Salesforce independently chose that pattern and started to use that to dynamically inject sidecars in Kubernetes workflows. This worked well until we realized that each of the teams was writing the exact same source code. There are same set of unit tests, same integration tests and producing the same Docker image. Writing the same health charts as well. Debugging the same problems in Kubernetes cluster. At that point we realized we should take a step back and see if we could derive a common pattern. We discovered that each team was using an annotation on newly created parts to trigger the injection of one or more sidecars. Each team had their own annotation namespace, sometimes one or even more and an annotation trigger. If the annotation was present on the part that meant it was a target for injection. So we looked around in the open source world to see if someone had already solved this generally. We did find at least one open source project, but it did not fit our situation particularly well. At that point we wrote a spec for what an ideal sidecar injector would look like. Dropped some code into a new repo and the generic sidecar injector was born. So at a high level, the generic sidecar injector is a mutating web mission controller that allows injection of additional containers, init containers and volumes at the time of part creation. So how do you use it? It basically uses a general configuration that consists of two parts. What needs to be injected and what triggers those injections? What needs to be injected is basically called sidecar configuration. What triggers is the mutation configurations. Separating out these configurations allowed teams to specify multiple sidecars and multiple mutations and independently choose which mutation injects which sidecars. This loose coupling supports different team structures such as if one team is supporting multiple sidecars or each team is supporting just one. So in the slide you can take a look, sidecar configuration is showing two sidecars that needs to be injected. The mutation configuration is showing two annotations and it also maps which annotation will trigger which sidecar injection. So locking will inject sidecar one and the monitoring annotation will inject sidecar two. Let's go move on to the next slide, yeah. Cool, so some of the features that we talked about in addition to containers, we support init containers and volume. And supporting multiple mutation config allows you to independently choose which mutation will trigger which injection from the sidecar. One of the interesting features about the generic sidecar injector is that it not only supports sidecar configuration using environment variables that is the native way, but it also allows you to specify sidecar configuration using Golang templates. Hyatt will show you a demo of how that actually works. So advantages, I mean, we already saw the advantages. It allows you to write code for mutation admission controllers and all of the teams can share the exact same code. So seven teams within Salesforce using the same code to solve multiple critical infrastructure sidecar needs. Some of the examples are monitoring, logging, certificate rotation, image signing, et cetera. So inner soul sowing is avoiding duplicate work, avoiding reinventing the wheel and avoiding repeating the same mistakes. So let's look at the demo now. Thanks, Mayank. Okay, let's go to the demo. So I will just move this thing here. It is appearing, I mean, so let's see. So here locally we have a Kubernetes cluster setup and you can see that it has a couple of pods running, right? So one of the pods you can notice is the sidecar injector pod, which is running in the sidecar injector next case. So this is the pod that is running our generic sidecar injector. So for the purpose of this demo, we prepare two folders. One is demo one and the other one is demo two. Demo one folder shows the demo how to do config-driven injection and the demo two shows in addition to injecting based on the config-driven injection, it also shows how to configure your sidecars based on the annotations that are coming from the pod. So let's go and dive in. So we are in the demo one folder. We have two files here, config-map.yaml and pod.yaml. Let's take a look at what is in the config-map.yaml. So you can see that it contains two sections. One is the mutation config.yaml and the other one is sidecar config.yaml. The mutation config.yaml contains the mutation configs, as Mike mentioned, and each of this mutation configs contains the annotation that is supposed to trigger this mutation config. And also it contains the list of containers that this particular mutation config will inject. Similarly, you can see that the second config, again, contains another trigger, different trigger, and it contains the list of containers that it can inject. Of course, the list of containers can be, you can specify multiple containers here to inject. So also you can see that there is sidecar config.yaml, which is just the description of the containers that you want to inject. In particular, the name of the container, the image that corresponds to this container, and the command that this container will run. So let's go ahead and apply our config.yaml because before the demo, we already applied this config.map.yaml. It shows that nothing is changed. Okay, let's go ahead and apply our pod. Before actually applying our pod, we can let's give it a look at what is inside pod. You can see that it is just a very basic pod specification. However, there is one critical thing to pay attention to. You can see that there's two annotations, logging and monitoring annotations, and each of them is enabled. And you can see that the key of this annotation matches exactly the annotation trigger we had in the mutation config. In particular, annotation main space slash annotation trigger. And you can see it is in the enabled state. That means that if we go ahead and apply our pod.yaml, this will create the pod. Let's take a look at the pod inside the main space. You can see that the pod is being created. Let's wait a couple of seconds to finish the crash. You see now pod is running. And you can see that instead of having just one container which was in the spec of the pod.yaml, you can see that it has a couple of containers running. In particular, it is three containers. Let's see what are these containers? We describe the pod one. You can see that not only it contains the container, main container that was in the pod.yaml, but it also has the injected container. In particular, it has monitoring sitefire container and the logging sitefire container. That is because if you remember, we had annotations in the pod.yaml which explicitly mentioned what should be injected and they triggered the injection by generic injector. Now with this, let's go to our second demo. So let's go to our second demo. When you've seen the second demo, we have again two fast config map.yaml, same structure in the pod.yaml. Let's open config map.yaml. You can see that it is same structure. However, there is one critical difference, which is the volume section. You can see as my mentioned earlier, you can inject not only containers, but also volume. In this particular, we are injecting volumes of one volume and we are mounting it to our monitoring sitefire, which means that any place where monitoring sitefire is injected, this volume should be mounted to this monitoring sitefire container. However, one other thing to mention is that you can see that the secret name from which this volume is derived, it doesn't contain static value, instead it has a suffix which is basically templatized. And you can see that the templatized value comes from the annotations of the pod. This is how we achieve templatization based on the pod's annotations. The idea is that later when the pod will have the chance to specify this annotation and provide any value it wants for the secret. So now let's go ahead and take a look at the pod.yaml. Again, it is almost similar to the earlier pod.yaml that we had for demo one. The only difference is that now it contains, not only it contains the triggers for the injection, but it also contains this annotation that matches the annotation we had in our templatized config. And you can see in this case we have demo two, which means that the result we are expecting is that the secret name will become aws-im-value here, which is demo two. So let's go ahead and apply our config.yaml. Yeah, we apply it unfortunately at this point we don't have the future where pod automatically reloads the config.yaml, which means that in order to reload the new config.yaml, we need to delete the pod and recreate it. So let's go ahead and take this pod and let's delete it. Injector was deleted. The new pod of course, Kubernetes will do its magic. It will again create a pod. And in this case it will pick up the new config map. So the pod is running. Let's go ahead now and apply our pod.yaml. Okay, the pod two is being created. Let's go ahead and do watch. So let's wait again a couple of seconds to see the pod being created. Okay, the pod is created. So let's go and see what is in the config config of the pod, the demo two. You can see that again, it has three out of three containers, which means that config driven injection worked. However, it has one other thing, which is in the monitoring sidecar, it has the following volume mounted as we mentioned in our mutation compute. It has the volume mounted and the value of the secret comes from the templatized version, which is AWS IAM demo two. This is what we exactly expected. This means that the value of the secret came from the annotation that was specified on the pod level. So to make sure we are not missing anything, let's go ahead and do executing to the container and let's see the value of our secret. That's it, it works. We see the secret and that means that everything worked as we expected. That's it, now I hope you guys enjoyed it. This was the two demo that we prepared. Feel free to go and try it out and give us feedback. For trying it out, we provide a couple of links here. We provide contribute link, which just simply points to our source code. You can go ahead, take a look at source code, feel free to suggest changes. And we have the block, which contains the info that we provided in this demo and more. So then you can also go to try out an example, which contains the examples that we showed. Again, we encourage you to try it out and tell us the feedback. Tell us your feedback. And of course we have, there are the following hours, which are basically hours we are available. Please feel free to come by and say hi. And yeah, just chat about this and provides our feedback. We'll be happy to meet you. Cool, thanks Hayat. Yeah, I think we'll be happy to answer questions at that time and note that the timings are in EST. So we're happy to answer questions related to the injector as well as any contribution related things. We welcome any contributions to that. So thanks all of you and have a happy Chukkaan 2020. Bye. Thank you everyone.