 Welcome to the CNCF Webinar and Production Ready Kubernetes for Python Developers with Pulumi. In this brief but insightful session, we will explore the powerful combination of Kubernetes and Pulumi to create robust and scalable applications. As Python Developers, you will discover how to leverage the features of Kubernetes while harnessing the flexibility and simplicity of Pulumi for infrastructure provision. Join us as we dive into the essential of Kubernetes and demonstrate how Pulumi enables you to define your Kubernetes infrastructure using familiar Python syntax. By the end of the session, you should have a solid understanding how to utilize Pulumi to streamline your development workflow, make your applications production ready on Kubernetes. Join us now. Shortly to my person. My name is Engindy. I'm working at Pulumi as a customer experience architect and I love everything about cloud transformation and enablement. I really enjoy to continuous everything from CI to CD. Everything should be fully automated. You will find me on different social medias also on GitHub. So feel free to follow me if you would like to know more about infrastructure as code and cloud native in general. Let's make an introduction about Pulumi for the people who may don't know Pulumi. So we define Pulumi in three different pillars. We have our build, our deploy and our manage area. In the build area you already may see some familiar icons of programming languages you know or you may already use. In this case, we're going to talk about Python but as you can see Pulumi offers much more support for various range of programming languages. So you can, if you're already familiar with TypeScript, for example, you can write your Pulumi code in TypeScript. If you're a Go developer, feel free to use Go or here comes the interesting part. Inside an organization, you can maybe use different languages to achieve the same goal. So the database team maybe work in Python. The application team works in Go. All of them can create their own infrastructure part or participate this to, for example, a developer platform. Integration in IDEs is completely available. So you can use, for example, VS Code, get all the intelligence highlights through your language system and autocomplete. Everything's there. The next part is the deploy part. Pulumi supports a huge variation of different providers starting by the cloud providers and heading over to different SAS services or services in general. The last part is the manage part. Pulumi integrates very well with your existing CI CD infrastructure. So you can use your existing Jenkins, for example, to execute Pulumi programs. You can use GitHub actions, but you could also use Kubernetes, for example. Pulumi also integrates with existing Git reposts to Git systems, for example, GitHub, GitLab, Bitbucket. Everything is possible here also. So as you can see, it's very easy to integrate Pulumi in your existing infrastructure, and it's all up to you to create cool programs. Let's talk a little bit about the core features Pulumi offers. So we talked before any cloud, any language. This is something you saw on the slide before. You see here also that most of the cloud providers are supported. Next point is Pulumi has a state management, so you can create a state will be created from your deployment and reconciled against the state. Pulumi offers inbuilt secret management, but it's easily replaceable also with the secret management of your tool. If you, for example, prefer Azure secret manager. Next point is that Pulumi always generates your preview plan. So you see any changes to your infrastructure and can react on this. This comes very, very handy if you want, for example, create a pull request with some new changes. You will run the preview plan and you get instantly feedback of any possible breaking changes or any changes which may interrupt your service. Next part we already mentioned this before is the CICD integration. So Pulumi integrates very well with existing CICD systems. Next point, Webhooks. Pulumi can communicate via Webhooks. You can execute your Webhook. You get information out of the Pulumi system. On top of it, Pulumi offers a really, really large REST API. So you can use this REST API to programmatically, for example, get the information out of Pulumi or execute specific actions inside Pulumi, the Pulumi cloud service. On top of it, Pulumi offers an automation API library, which comes very, very handy if you think about starting to programmatically use Pulumi. So inside your Go application, TypeScript or Python application, you can now execute programmatically. Pulumi programs gives you the ability to create your own backend service, your own CLI, your own Kubernetes operator even. So here the sky is the limit and you can really, really integrate Pulumi the best way for your organization. On top of it, dashboards and reports, which comes very, very handy if you want to get out more information. How many resources I'm already using, how many EKS clusters, for example, my whole organization is using and this helps you to accumulate the data and maybe also execute some reports out of it, getting some, for example, financial information about your cloud spend. Let's head over to the Pulumi high level view. I found this very interesting to have a good understanding how Pulumi is working. So this is the Pulumi architecture. I'm going to explain this in more detail. So for now, let's start. The language shows the most important part. You choose which language you want to use. You include the SDK, depending on the language you're going to use. And this is the part where you start to program now. You start to create inside your language, you create now the resource. This interacts with the CLI and the engine. So the Pulumi CLI is actually also the Pulumi engine. Somebody termed this as Pulumi is actually a microservice. Several microservices ended up accidentally on your local hosts. That's the fun explanation. But this is the moment you use your language shows, you start writing your Pulumi program. The CLI will, depending on the provider, you choose start the provider as a microservice. And here we come also to the providers. So if you use AWS, for example, Pulumi will start the AWS provider and then use GRPC to communicate with the provider translating your language description of the resources you want to create into GRPC called execute them against the provider. Last but not least is the state backend Pulumi, as we mentioned before, saves the state. And you have different possibilities to handle the state management. You can host it on your own. There is no, no problem with this. So you could use, for example, an S3 bucket, you can use a flat file system. There are many, many possibilities also to store your own state management, or you could use the Pulumi cloud service, which is free for open source users. And you don't need to think about handling the state files. And now comes the worst part, state files on a self-managed system can be accidentally deleted, can be corrupt. So you end up taking this ownership for a state file. Me, for example, I always encourage people to say, okay, do you really want to take care of the state file on your own? Or do you want to give it to somebody else to say, hey, take care for me? I don't want to think about the state file. Next part of our high level view tour of Pulumi is how Pulumi looks like from a project view. So everything starts with a project. A project contains the Pulumi program. This is where literally spoken is the folder where your Pulumi.yaml file lies in the Pulumi.yaml file defines the runtime, defines the name of your program, the description text, for example, and so on and so on. So this is what we say a Pulumi program. Next of it, Pulumi has this concept of so-called stacks. Stacks are different instances of your Pulumi program. So think about this, and this is how people mostly use this, using stacks to represent your different instances. So you have a dev stack, a QA stack, and a proc stack. So now comes the interesting part. What you can do with the stacks, not only you can differentiate the different instances, you can also overwrite some of the properties, for example. So for example, your dev stack needs a database URL. But this is most likely a different one than you use for QA and proc. So here you can set the variable database, for example, as a config parameter, and then overwrite it in the different stacks. This is very, very interesting and very helpful if you want to write your Pulumi program once and just change the properties for the different instances. And then we head over to our smallest building blocks, the so-called resources. The resources is where you start to put the pieces together and define your whole infrastructure to say, okay, I need a VNet, I need a Kubernetes, I need a container registry. You can start to put them all together. Interesting part is, again, you can, inside a Pulumi program, use different providers. So you could use inside a program an AWS and an Azure provider. For example, you want to use Azure container registry. So you could use the Azure provider to create your Azure container registry, but then continue, for example, in AWS, create there your EKS cluster and then just combine them and say, okay, I configure my EKS cluster that it's fetched all the container registries from the Azure container registry. It's all possible. And here comes also one of the important features. You can use the inputs and outputs for every resource. So every resource has the possibility to set different values. For example, the size of your container registry or the size of your Kubernetes cluster, the amount of nodes you want to connect. This can all be controlled via the inputs. And now comes the interesting part. Every resource also generates outputs. And we talked about the different stages. We talked about the database URL. So when you create a database inside your Pulumi program, it will generate you an output with the database URL. And this one you can use now to feed it into another resource. For example, in your HelmChart resource, you could use your HelmChart resource and set the value there with the database URL. This is very, very interesting to use. And the last point is you can also use different projects, the output of different projects as input for another project. So think about the database team created a project for giving, for creating a service of databases. Me as an application developer, for example, I'm not interested in how the database team implemented their service. What I want to know is, hey, this is the size of a database. I would like to have, for example, I want to have a large database or I want an MSQL database or I want a Postgres database. So these are properties I want to set. I said I want a Postgres database with a large specification because I want to save huge data into it. These are the inputs I give into the project. And the project gives me then the output, for example, the database URL, the specific admin user and so on and so on. These are the values I'm interested now as a developer to ingest and use inside my application, for example, or inside my deployment of a Helm chart on a Kubernetes. So you see, we can create here different layers and see also that different parts of the organization owns different stacks, for example, but everybody works nicely together using code, for example, and maybe publish this service also to an internal developer portal where people can just consume this. So the inputs and outputs will be, for example, text fields I can use to choose what I want. Click on the button in GUI, for example, and then it will get provisioned for me and I can use them in a different Pulumi program. We talk much about templates. So let's deep dive into the Pulumi templates and say, okay, what are Pulumi templates? Pulumi templates are really the perfect way to deploy your infrastructure with best practices in mind. So whenever you're in your Pulumi CLI type, Pulumi Nu, you will get a huge list currently, 250, I just checked it, 215 templates you can choose from. And then you can start saying, okay, I want to create an AWS RDP, for example. So you just choose the proper template and fill out the values inside the template. There's a prompt when inside a template, some parameter is defined has to be set during the initialization of the template. The Pulumi prompt will ask you about this. And now here comes the next good part. You can create also your own template. We just talked about this different teams provide different services as code. So nothing wrong on that the service, for example, creates a template for this and say, okay, we from the database team, we have a database template for small databases, we wrote them into the different languages, our organization is supporting. And then you can say, okay, I just say Pulumi Nu, point to the git repository where the template lies in, and then fill out, follow the prompt. And then I will get my best practice database from the team. Okay, templates we're going to use also in our demo. So that's very interesting that you will see this also in action. As I said, now it's time for the demo. I think we covered a lot of the theories and I'm 100% sure you want to see how Pulumi is working in action. So let us ask our demo gods to say, okay, what should we use as a demo? So let's head over. Okay, and the spinner goes on. And as we knew, we want to use Python and we use Python, AWS and EKS in our demo. And now comes a little spin. I'm going also deploy a Minecraft server on top of it. So we not only use Python to execute a Pulumi template to create as an EKS structure, we also deploy on top of this EKS Minecraft server. And this is going to be awesome. So let's head over to the demo, I say. So let's start with our demo. So for this, I created a new folder called CNCF webinar Pulumi. And here I'm going to use the AWS Python template to start our Pulumi journey. The project name, I did it like it is, it will take the folder name as a default, I let the default description. And of course, I'm going to use prod as a deployment stack. And I choose as a region EU central one. Oh, I have a little spelling error inside here. We can correct this. That's not a problem. Just head over to our Pulumi prod yaml. And then we change here the central. So as I mentioned before, we're going to use the EKS provider and the Kubernetes provider. We just need to add this two providers into our requirements text and then let Python get this new package. So that's completely fine. And now we see in our main Python source file that there are already two, one resources defined. That's our bucket that comes from the template. We don't need this here. So I can delete this all. And we're going to start with our infrastructure for our networking for our cluster. So I'm going to import this files now here. And I'm going to start to create the VPC subnets we need to use for our cluster. So here we see we create a VPC internet gateway, a route table, two subnets to have availability zone available, and then the appropriate route table for this. Next one is now we create our EKS cluster. And for this, we're going to use the EKS provider. And the EKS provider is very nice because it already created some kind of abstraction for us. So we just need to configure some parameters like the size of the nodes and how many I would like to have. And if there's some auto scaling going on and much, much, much more I can configure here. So here it's already configured. And the last piece I'm going to add now is the Kubernetes provider. So we're going to create the Kubernetes provider. So here we can see also that the Kubernetes provider set. So this is again one thing I already talked before. In one Pulumi program, we can have multiple providers for different cloud providers. So in this case, I created on top of the AWS provider, I created a Kubernetes provider because I'm going to deploy Kubernetes resources. I could create different Kubernetes providers depending on the Kube config. For example, if I have Kube configs limited to specific users with specific rights, I can just create an additional provider to say, okay, this is team A, team B, and then I deploy the workload according to the Kube configs. For example, I could create multiple Kubernetes cluster and multiple providers. You see everything is possible here. And it's really up to you to put the things together for your needs, for your problems, you're going to try to solve here in our simple case, one cluster, one Kube config. And then the last bit is we just integrate now the Minecraft chart. So that's it. We talked about this Pulumi integrates very well with existing solutions. So here in our case, on Helm chart, we can easily integrate the Helm chart into our Pulumi Python program, which is to create here the resource release, and then fill out all the forms we need to have to get our Minecraft server up and running. Then I can say Pulumi preview. And I will get the results of Pulumi preview. This is taking a little bit time. Everybody knows that creating a Kubernetes cluster can take some couple of minutes. So I already prepared something here. So let me switch to the place where I already created a Kubernetes cluster. So now we see same code. I already applied the Pulumi app. I can do it again, say Pulumi app, and it should give me now an update and see there's nothing changed. That's completely fine because that's what we expect. I didn't add anything additional to it. So that everything is still as I deployed it before. And now we can check also for if our Pulumi, if the Minecraft server is up and running because we did not allocate it now a load balancer to it. I just want to see via port forwarding that I connect to my Minecraft server. So in this case, we just need to add some little piece to it because I want to get hold on to the kubeconfig file. So I can use Pulumi export. And I can export the cluster kubeconfig file to kubeconfig. So in this case, I just say please apply the changes again because now we have a change. We want to export a file. So as you can see, he already gives me the feedback. I can say yes. So it will roll out now the changes. No need in the infrastructure change. So I saw there is a change. There will be an additional resource created here. In this case, it's just an output. And now I can say Pulumi stack, Pulumi config. No, no, sorry, Pulumi stack output. So I can say Pulumi stack output. The name of the output I would like to see here in this case is the kubeconfig. And I can have it here as a kc. I just accidentally created it here as kc. So and now I can export the kubeconfig to the kubeconfig environmental variable. And let me start k9s here. And I see that my Minecraft server is completely up and running. So that's completely nice. And let us see that it's also running, checking the pod. Yep, everything is spawned. So then we try also to connect to our Minecraft service. We go to the Minecraft service service, activate the pod forwarding. And now we should be able to connect to our Minecraft server. And for this, I start the Minecraft client just a second. So we start our Minecraft client and change the server address to our localhost1127.0.01, refresh the server and see that it appears. Yes, it appears. This is our Minecraft server we just recently created using Pulumi. Let's hop in and see that everything is working fine, that we really can connect to our Minecraft server. And yes, we are inside Minecraft created. We are Pulumi in Python. So that's it for our little webinar on how to use Pulumi with Python to create a EKS cluster and run workload on top of it here. In this case, a Minecraft server. I hope you enjoyed it and looking forward to see you again in one of our next talks. Bye.