 Hi, welcome to all of you. I'm Marcus Birkeloon, I'm based in Spain and I work at VMware, which I joined via the acquisition of VINAMI in 2019. Today I'm going to talk about some of the lessons we have learned at VINAMI while deploying traditional web applications on top of Kubernetes. This talk is structured in the five parts. First, we will have a brief introduction. Next, we will talk about some of the different steps you will follow if you want to deploy a traditional web application on top of Kubernetes. Afterwards, we will give some tips and good practices and discuss also some particular challenges you may find. And finish the talk with an example. At the end I will be here to answer any questions you may have. First, I wanted to talk a bit about my company. VINAMI, which is now part of VMware, is a catalog for web applications, run times, frameworks and more, that can be deployed in multiple environments, platforms and formats, such as continue images or help charts. We have more than 10 years of experience deploying web applications on these environments. In fact, our catalog contains almost 100 traditional web applications, and more than 20 of them can be deployed in Kubernetes via hand charts. Let's talk a bit about traditional web applications. There are so many of them, incredible popular ones, less popular ones, etc. This is just a small portion of the ones that we support at VINAMI in one of those platforms, and you will easily recognize more than one logo. They are not designed to work in cloud-native environments. They date to be for that with this anything. So how can we differentiate traditional and cloud-native applications? I have compiled a few basic requirements that we will expect any Kubernetes native apps to meet, so they can be deployed without any headache. Traditional applications, however, don't usually meet one or more of these, requiring work routes and hacks. In some cases, it might even be impossible to solve these problems. Let's talk about them. First, the application's process may be stateful, share researches between them. Next, there may be no separation between application code, data and configurations, not allowing to specify deployment inputs without relying on configuration files stored in the local disk. The application may not be ready to be stopped at any point of time, or require some time to boot up. This usually happens a lot in Java applications, such as Jenkins. The application may not support horizontal scaling due to design limitations. Usually this happens when it relies on the local disk for storing data. Finally, the app should specify an exact list of dependencies that it has been testing working with. And then, ideally, with the standard way to install them. For example, with a package manager, such as NPN, to avoid any issues when running the application. We have also created a GitHub res procedure for this talk to complement with examples and more detailed explanations. It is on github.com. Slash Bitnami Labs. Slash Deploy Web Apps and Kubernetes. You don't need to copy the link right now, as we will be referencing it throughout different parts of this talk. Now, let's briefly revisit how you would proceed if you wanted to deploy a traditional web application on top of Kubernetes. Here, you can see a list of basic steps you would need to follow if you wanted to deploy it. You would need to first build a container image, build a ham chart, and then deploy and operate on Kubernetes. For this talk, we will be mostly focusing on the building process for container images and charts. Let's take the base step for running an application on Kubernetes. Let's start with the building of container images. When we talk about container images, we will refer into OCI compliant images. These can be random docker, podman, or Kubernetes, for instance. A particular thing about container images is that you can run code at two stages. At build time and at runtime. First, you can run code when you build the image via the docker file. It includes a series of steps to define your image, which in this case means you can build and run the application. It is based on layers. Each command in the docker file is executed on top of the previous layer. You can specify inputs, but they will only be used when building the image. In the example on the right, you can see how an example docker file would look like and how you would deploy it. Its format is really simple. The runtime is specified by an entry point script. Here, you would run scripts to install and run the application. You can specify external inputs via environment variables or mounted files. Let's talk about docker file design. It should contain all the steps to install the application dependencies, copy application files, and prepare it so it can run as well as making it secure. Now, it is heavily recommended that you also optimize the image size and also ensure that installation can be performed with as much stability as possible. You would also specify runtime configuration defaults, such as the default entry point, default values for inputs, etc. Now, about entry point design. Ideally, it should contain steps to validate inputs, persist the application data, start the web application itself with the focus and thinking the latest amount of time to deploy, then other things that would be nice to have, like logic to configure the application via external inputs, waiting for external services to be available so the application doesn't throw any error, or other post-installation features, for instance, to automate installation in the database. Next, let's talk about Helm charts. Helm is known as the package manager for Kubernetes. It's the most popular way to deploy web applications on top of it. A Helm chart contains a collection of Kubernetes resource templates that can be extended via inputs during deployments. It supports complex architectures via subcharts, meaning that a chart can include another chart. To the right, you can see a basic file structure of a Helm chart, and below you will see the steps to deploy it into Kubernetes. About Helm chart designs, there are so many things that you might want to support. Some things it will mostly always include for your traditional web applications. For instance, deployment for application containers, ingress with TLS word, persistent volume claims to store your application data, or secrets to storage credentials. Other things you should consider supporting will be supporting the database in a subchart, or using an external database for it. Metrics, ground jobs, customer resources, etc. Finally, let's talk a bit about build and release pipelines. Why would we want a release pipeline? Well, it will allow you to perform tedious manual steps in an automated way. It should contain the steps to build both the container image and Helm charts, plus the automated testing, releasing, or publishing of both of them. One of my work colleagues presented last year in KubeCon talking about the internal Bitnami build and release pipeline. If you are interested in the topic, feel free to check it out. It's on YouTube. Finally, I also wanted to mention that there are a few alternatives if you want to implement your own CEI or CD pipeline. All of the examples listed here have a free tier if I'm not wrong, and there are also open source alternatives like Jenkins, which we use at Bitnami. Now let's talk about some tips and good practices when building container images and Helm charts. First, you should consider running containers as non-root users. This is because it allows to protect from privileged escalations and access to non-readable files. For instance, if a malicious user gained access to your application's file system, it would greatly limit the stuff they could do to make it behave in an undersirable way. There are different alternatives you can follow. Running a non-system user or an existing user with an arbitrary user ID or an arbitrary user ID and an unknown username. The problem with the first two is that they are either insecure or they are not supported in all platforms, and they do not follow the best security practices and recommendations. For these reasons, we heavily recommend running containers with an arbitrary user ID and an unknown username. It might not be compatible with all apps, but you can work around this with NSS wrapper. Another side effect of having no username is that the shell prompt may look a bit ugly. This doesn't mean it is broken. In this case, any writeable file must belong to the root group because any username that has no name belongs to the root group by default. Now, let's continue talking about permissions. It is important to set this at build time, as non-root images may not only lack the privilege to change them at runtime, but also because it may be slow. At build time, we have all the time of the world to do so. So what files should we make writeable? Well, any file that the application requires to modify at runtime. If there is a clear separation between application code and data, then only data files will need to be writeable. You will need to identify the list of files and folders that are writeable when you create the image. It might be the application requires access to the entire source directory in case you will need to make all files writeable. Finally, I recommend to check the Docker security cheat sheet from the OWASP foundation. It contains some good security topics that you should review when creating your content images. The most relevant one will be with respect to read-only file systems. If you want to support that, you need to identify the list of application files and folders that need to be writeable. Like mentioned above, and create an appropriate volume for those. Then, populate the volumes at runtime if applicable. Next, there are other topics like using Linux security models, such as app.app-armor, second comp, or sc-linux. Another important factor when creating your content image is the image size. So we know that content images follow a layered format. This adds good things, but one of the things that can go wrong comes when we modify a file in different layers. It gets counted twice. Even changing the file mode or ownership will have this effect. And why would you want to fix this? It is important because it will affect bandwidth usage and deployment time if the image is not stored locally. It will need to be downloaded. So we can be talking about several gigabytes of data transfer here. There are some strategies you can follow to easily achieve an optimal image size. First, by using multi-stage images to build your application. This is especially useful for applications that need further steps to be built, with specific build dependencies. The next options consist of strategically implementing your Dockerfile, grouping any actions that may modify a file more than once, removing temporary files, etc. and all in a specific order. This may lead to Dockerfiles that are hard to read. The final thing you should consider, if the previous two are not possible for your case, is to use Docker's squash feature. It is some sort of nuclear option, as this has some important limitations, but it's up to you to consider if there is no other way. I also wanted to talk about the useful tool called dive. This tool allows to inspect images for big files, so you can check where to start optimizing the image size. You can see this example of a WordPress image, which is more than one and a half gigabytes in size. When we optimized it, we were able to take it down to less than the half of the size. That's a lot of space that the user saves, but also in deployment time, as it does not need to be downloaded. Now, let's talk about some good practices creating container images. We have collected the guide on some of the best practices it should follow when writing a Dockerfile. Find a link to it in this page. The main recommendation consists of running only one process per container. We will discuss different ways to work around this problem later. At build time, you would ideally run heavy and time consuming tasks to boost throughout the application and avoid those at one time. You should also be aware of any possible reproducibility issues when building the image, for instance network connectivity issues. As for maintenance, it is highly recommended to rebuild the image periodically. This could allow you to minimize the response time in case of a security incident affecting a dependency of your application, or even by detecting a potential build errors before a rebuild becomes urgent. A mistake we made at Biennami in the past consists of using an external tool to initialize the application. This had some issues with respect to visibility in the back. But the main issue was that in some environments, it had such a huge memory impact that it will cause issues when initializing in Kubernetes, due to setting resource limits. Other things you should consider at this stage will be to support for multi-arc, such as EMD64 plus ARM, or choosing your own distro among other things. As for charts design, I linked here a guide on some of the best practices creating hand charts ready for production. Find also a link to the public hand chart template we're using at Biennami when we add any new solution. I would mainly recommend that you configure it to deploy non-root containers by default and allow operators to extend this to their will. Find some examples here, for instance by loading to specify custom container commands or visible props. Integrate with logging and monitoring platforms such as ELK or Prometheus. Add diagnostics features to make it easier to debug problems in deployment in case anything happens. Now let's discuss some of the challenges you may find and how to solve them. If you want your application to work properly in Kubernetes, you need to ensure that persistence is properly set up. If not, you may lose data after the container gets re-created. As a general rule, you should only persist data files and folders that need to be writable by the application at runtime. The application separates data and code. In this case, it could make our task easier. Beware that there are many applications that don't do it. This approach is faster when you do the first initialization because it needs to copy less files. Just imagine if you had to copy an entire multigrid application via the network to a different volume. A last point of favor in this is that it makes it easier to upgrade the application by just swapping the container images. If the application doesn't have any code and data separation, then you will need to persist the entire application source code. This can be slower than the previous approach. Think of the example we mentioned. The other downside of this approach is that it forces the operator to deal with any sort of maintenance and upgrade tasks. Which approach you choose is up to you. You should consider the pros and cons of each one of those. Now let's talk about support for automatic updates. We understand automatic updates as the process where the operator does not need to manually run any upgrade command or wizard to upgrade the application. Note that before you make any upgrade, it is essential that you backup your database and scale down to one node if the application is running through multiple nodes. If the application separates code and data, like I mentioned above, it makes it easier for the operator to upgrade by just swapping the container image version. Another way this can be implemented is if the container is, if the application itself, supports some sort of automatic upgrade process. Like WordPress, where the operator doesn't even need to worry about clicking on the upgrade button as the application does it automatically. The problem with this method is that this requires the application itself to have right privileges to modify the application's source files. This is not ideal with respect to permissions. It also makes it more complicated to roll back in case any error occurs. If the application does not provide any of those features, that is code and data separation, or an automated UI update tool, then the operator will be forced to perform a manual upgrade process on their own. Another issue you might find is that your application requires multiple services to work. But when the general rule is not running more than one process in a container, then what do you do? Well, there are two main problems we have identified related to this. First, when the application has a set of external dependencies or services that need to share data with the main component or process. Like for instance, via the use of workers. In this case, you should check if it allows to do that with an external service. Like Redis. If not, you can set up a synchronized or cached file system such as NFS with a downside of a notable write performance. The next problem comes with respect to front-end and backend servers requiring access to the same application files. In this case, you should check if there is any way to host external resources via a CDN. If not, you can also rely on shared or cached file systems like we mentioned above with a notable impact on performance. In this case, you can also use a cached server like Varnish for caching front-end resources to help in the performance drop. Other cases you might find could be periodic jobs. In this case, Kubernetes offers a crown job resource definition that you can use to work around these issues. Or you may want to execute some pre-installation steps, apart from what is in the container logic. In this case, you can use an init container. The last challenge we will talk about is horizontal scaling. Ideally, your application will fully rely on external services and host no local data whatsoever, making it easier to scale horizontally. For example, you will have a web server to host application data, a different server for hosting temporary files, such as cached sessions, etc. Usually don't be a redis, memcached, and other types of servers. Then you will have a CDN for hosting the application assets, user uploads, most others. And an external load balancer as the main entry point from users to access the application itself. In reality, traditional applications require hosting local data, like we have mentioned before, and the solution to that may be using a synchronized file system. Even with that, the application may not support it due to extra limitations. In this case, all you can do is optimize it. For instance, you can use a cached server like varnish to reach the load from the application services, optimize the application configuration to your needs, or even scale vertically if needed. Now that we have seen all the different steps involving deploying an application in Kubernetes, including how to build a good container image and a good hand chart, let's see a real-world example. We are going to describe how to deploy a very popular application, in this case WordPress and Kubernetes, following some of the best tips and practices mentioned in this talk. First, let's see how a component diagram would look like. In this case, the user will connect to the HTTP port of an NGINX container, which would then communicate any request to PHP-FPM via a proxy and using the MariaDB container as a database. The only container we will focus on for this talk would be the one that executes the WordPress application. In this case, the PHP-FPM container. This is an example of a real Dockerfile that could be used to build and run the WordPress application. Don't be scared. We will be describing it step by step. You can see here that we are using a multi-stage build. In this case, to build the application in a separate container and save space in the overall image size. We are using the same image as the runtime to save space as well. We are installing build dependencies, in this case for the cURL command, extracting the application, and adding any customizations on top of it. Next, we are defining the new image, the final image, using a minimal container image with a minimal size. We are adapting it to our needs, installing runtime dependencies, finally installing WordPress from the multi-stage build, applying any runtime configurations, persisting the WB content as the data folder for WordPress, and running everything as a non-root user. This container is using an entry point script that we are describing here. In the same way, we are describing it by step by step like before. First, we are setting bash strict mode and load functions, which is a good practice. Next, we are determining if the application is being initialized. If it is, then we will persist the application data files. This is only on the first one, it may take a bit more time than the usual, but in successive runs, it shouldn't do much, it should be almost instantaneous. Another step that we do in the first initialization is validating inputs, running custom commands, etc. Finally, we start the application. And that's it, that's basically a container image. Now, for the chart. Now that we have a container image, we want to deploy it on Kubernetes. The chart includes a lot of files, so we will not be describing into detail, because we lack the time right now. But we have collected two examples. One of those is the Vindami WordPress chart, which is the last in Apache plus ModPHP approach. Some basic features is that it uses MariaDB subchart as the database, but also allows to specify an external database. It also supports catching via a WordPress plugin, plus maincatch. We have also created for this talk specifically an NGINX plus PHP FPM example, basing on the Vindami WordPress chart. Refer to the link to the GitHub repository for this talk and check it out. So thanks so much for attending, I really hope you enjoyed this session, you learned a lot, and I encourage you to ask any questions and share any feedback. I would love to hear from you. See you!