 Good morning everyone, I hope you are enjoying the day today and I again welcome you all to this amazing event open source summit Europe at Dublin Island. My question to you all is how have you guys previously worked with certificates in Kubernetes or have manually managed certificates in Kubernetes, like how many of you? Yeah, I can see very few of you. So today we have the session that I'll be talking about security with certificates in Kubernetes. So let's get started. So this is Akshath and Myself Nathi. So we are working as a member of technical staff at VMware and we have recently graduated from SRM Institute of Science and Technology Chennai, India. We have done our bachelors in computer science. So hello and welcome, we are the speakers for the session. Let's start the fun session. So here we have the agenda for today. First we'll be covering about the certificates basics, like what are certificates and how two parties use certificates to interact. Then we'll be covering what is the need of certificates in Kubernetes, how Kubernetes manage certificates and stuff. Then we'll try to understand what is a certificate authority, how it signs them, issues them that we'll try to understand. And also I'll be explaining the workflow diagram, how the certificate signing process takes place. Then we'll be covering the certificate generation, how certificates are generated and then followed by the certificate generation APIs in Kubernetes. Then comes the most exciting part, that's the hands-on demo that we'll be doing. And then comes the final part, that is the certificate bootstrap and rotation where we'll be learning how we can rotate our certificates when it is near expiry. So most of you will be knowing about what are certificates, why do we use them, but for some newbies out there, we'll explain it again. So certificates are none other than a way of authentication between client and the server. Take the client and the server authentication, the client has to authenticate the server and the server has to authenticate the client so that both parties are contacting the right one. So we can authenticate them. So certificates enable authentication between different components in the Kubernetes cluster also. So in the previous slide we saw what are certificates. In this slide we'll see what is the need for certificates in Kubernetes. So this is the Kubernetes cluster, this is the master node and this is the worker node. So there are different components in a Kubernetes cluster as it is a distributed system. So there is API server, scheduler, controller manager at CD and the kubelet. So the API server is the main point of contact between each and every component inside a cluster. So it has to communicate to all the components. So it needs a certificate or a validation that it is talking to the right component. So we need certificates in a Kubernetes cluster. Now cluster certificate authority certificates and signing them. So certificate authority is a trusted party with ensures the authority for entire Kubernetes cluster. All these certificates are signed, managed and rotated by the cluster certificate authority. So take it as like in this diagram we can see that at the top part is the certificate authority and these two are client and the server. So cluster certificate authority will give certificates to both the client and the server so that they both can trust each other. They both can trust the third party which is certificate authority and it will issue the certificate to them. So that they can validate each other. Now comes the workflow diagram. We will try to understand what is the complete workflow, how it works, how certificate signing takes place. So we have a pair of keys that is the public key and the private key. So using the public key and the tokens and the CSR certificate signing request, sorry the certificate is the identity info. It goes to certificate authority through a CSR. CSR stands for certificate signing request and once the CSR reaches to the certificate authority, the certificate authority validates the certificate and also the token and after that it issues the certificate and also signs the certificate and this is the signed certificate that we have received in the end. Now we have learned about certificates, how certificates are generated, how what is certificate authority and stuff. Now let's create our own certificate, how you can create your own certificate, how I can generate my own certificate. So there are two generate certificates, there are many tools but I have written three open source tools that is EZRSA, OpenSSL and CFSSL. So these are the commands that we can run. You can also find these commands in the official documentation of OpenSSL. OpenSSL generate RSA followed by the output name of the key file that is domain.key and then we can request the CSR using the second command that is OpenSSL request and REQ and followed by the key that we have generated in the first command, followed by the output name that can be domain.csr and also coming to the third command, after we have created the CSR and the key, we will be using it to generate a certificate. So here is the third command that we will be running that is OpenSSL x509 followed by the key and the CSR that we have created along with the days for how many days it should be valid and the certificate name that we want the output. So then we have to create a private key and a self-signed root certificate of the class certificate authority. So we need a certificate for a certificate authority also. So we have to run this command OpenSSL request and the number of days for which we want the certificate to be valid. Then we have we can see the certificate in plain text using this command OpenSSL in the form of text. So we are writing text. This is the last command you can view the normal text format base 64 decoded format. Now certificate generation API in Kubernetes. Kubernetes also offers a certificate generation API which is certificates.k8s.io. This is the endpoint in which you have to hit it and then it will generate a certificate. So it provides a certificate which is signed by a certificate authority that we can control. So client creates a certificate request and then this certificate request is sent to the certificate, the certificate generation API in Kubernetes. So it is stored in a pending state before it is approved. So then the cluster admin approves the request and then it is issued till that it is there in the pending state only. And after that the certificate request certificate signing request is approved and the certificate is generated. So in this diagram we can see certificate creation and service to service communication. So the certificate authority is made by certificate signing request and the config file. Then the trusted device sends a certificate signing request and the certificate authority in turn sends the signed certificate and key to the trusted devices so that they both can communicate with each other using the certificate and the key. Okay, so now it's time for a quick demo that we will be doing and understanding how certificate works in our Kubernetes cluster. So here we have our Kubernetes cluster up and running and this is a mini, I am running this cluster in my local system that is a mini cube cluster and in this mini cube cluster we can see there is a default Kubernetes service that is running. So here we can see there is a default Kubernetes service that is running in the mini cube local cluster and like once this is created now you have a cluster up and running we will go inside the directory and create the required files for that, yes. So here this command is used to create your CSR, so for creating your CSR you need to specify the host, the first two are the internal DNS of the pods that we have to define followed by the internal IPs of the pod and we also have to give the key what algorithm it should use and the size and you can see in the output the CSR is generated. After running this command you will get two files that is server key file along with the server.csr file. So our CSR is formed after this, we will use these two and we will approve first. So here the CSR file is formed in the previous step now we need to apply it to our cluster right. So we will run the command QCTL apply and we can give the name as per our preference so I am giving it osssummit.default and I am passing it in base 64 encoded format. So I can see the response that it is created and when I do QCTL get CSR you can check that the condition is in the pending state. So the CSR is created and it is also transferred to the cluster but as she mentioned that when a CSR is created it is in the pending state until a Kubernetes admin approves it. So I will do describe CSR followed by the name I can see here it is in the pending state and these are all the details that we have for our CSR and also we have the signer name. Now cluster admin will approve it so we can run this command QCTL certificate approved followed by the CSR name so we can see we got the output that our certificate is approved. CSR is approved. So we can see it was approved. Now we have learned the CSR how all the things in our CSR work. Now let's create our custom certificate authority so that we can sign our own certificates. So in this we have given this the name here as the signer and we can see that certificate authority files are created these are the three files this is the key and followed by the CSR for creating the certificate authority. So these are created using this command and we will be using it to sign our own certificates. So and also for creating a cluster authority we also have to give a config file so this is the config we are giving the permissions or the usage what a cluster certificate authority can do followed by the expiry for what time the certificate authority should be valid for. So we need to pass the CSR along with this config yeah so here we have these three files all along with the config yeah. Now I am running it QCTL guest CSR this is approved we saw in the previous step now let's sign that using the certificate authority that we have created yeah we have to give the name here using this command we can sign so we can get signed certificate with serial number this so our certificate is signed now let's check again the status QCTL guest CSR okay yeah we have to also give the name yeah and when I send a signed certificate to the API server it is I receive this JSON response so this is the JSON response which is received after sending the request and the certificate to the API server okay so I am when now you can see like I have send the certificate to the API server and I can see that it is issued now it is approved as well as issued and we can also view the details and here I will be showing the yeah so this is the certificate that we have received this is the contents actual contents of the certificate that we have this but this is the base 64 encoded format so let's decode it and see in the plain text we can simply add this here in the command and yeah so this is the certificate in plain text format this is the plain text format these are the contents of our certificate that is downloaded like we can download this certificate use it in the Kubernetes cluster anywhere yeah well now once the certificate is issued it is signed by the cluster administrator it's done right now how to use it how I can consume it the things are done but consume how I can consume the certificate so for that I am creating a certificate here so Cube CTL creates certificate followed by the certificate server dot CRT that we have created this is a signed and approved one and also the key that we have to pass so here we can see the output the secret is created and we can also verify that using Cube CTL get secret we have a server secret uplisted there yeah so now let's use this secret inside a pod so for that I am running this command you can create a config map and use this yes yeah so let's come back to the slides so yeah a quick recap of the demo that we did so here we talked about how you have created certificate pair using the CFSL tool the two files that we got along with we saw that how you can approve the CSR how you can create your custom certificate authority we saw how we can sign them uploading to API object the response that we got and also you can download and use it in secret and the part so this was a demo that we saw now let's cover certificate bootstrap and rotation so certificate bootstrapping is nothing but when a client creates a CSR it is associated with a token as well so the token and the CSR goes together for validation like whether the CSR is like it is received from a correct source or not so to validate that we have we also pass a token and the CSR approving controller which is there it validates the CSR and also it validates and then process it so the after the certificate is generated we have seen that we can download the certificate now coming up next is the certificate rotation steps that we have certificate rotation is nothing but when your certificate is coming near expiration when it is going to get expired so it is known as a certificate rotation so by default the certificates are generated for one year so that like we don't have to renew it again and again so but we can also keep it as per Irish so like when you want to if you are using cube ADM then you don't have to manage certificate it will rotate automatically for you but if you are not then you have to manually do it in the cluster so using this certificate rotation technique we can we have to rotate our certificates so once the new certificate is available then we can use it to authenticate the connections in a cluster so to make the process automated we are using a cubelet to make the certificate rotation automatically so we can run this parameter rotate certificate which will control that the cubelet will automatically automatically renew the certificate when it is nearer to the expiration date so we can also write the certificate signing duration so that we can specify for how long the certificate are to be generated or to be issued for so yeah like that's it for today and do you have any questions for us you can reach out to us and connect on LinkedIn and Twitter if you have any questions do reach out to us and if you have any questions please do let us know thank you thank you