 Alright, good afternoon everybody. My name is Media Ghazi Zadeh, I'm one of the maintainers for Minicube. I'm a technical lead manager at Google and I have the honor of leading the Minicube team for the past few years, past two and a half years. And then here in this talk I want to share with you an overview of the maintaining Minicube in the past five years. And also I'm going to share 20 pro tips of using Minicube. And also some lessons we have learned in maintaining an open source project. I also have with me Anders Birkland from Gothenburg, Sweden. He's online. He'll be joining us in spirit and online on SlackChat. You can follow me on github or Twitter. I don't have that many followers, but I will only speak about Minicube and coding in this Twitter media dev. I'm curious by the show of hands, who has used Minicube here? Okay, that is almost everybody. And another question, who has known or used Kubernetes more than five years? Almost nobody. So I think Minicube is older than all of us, including myself in using Kubernetes. I myself did not use Minicube. I first used Minicube about four years ago. So I think when Minicube was started, I did not even know about Kubernetes at that time. So the previous maintainers for Minicube, Dan Lawrence started it, and then Thomas Sturmburg and Matt Rickard. But the current maintainers of Minicube, the people behind the Minicube emojis are these amazing people under Sharif Algamal, Stephen Powell, and Frederick Orogovic. And we are from diverse set of countries and companies. We have three Googlers and two non Googlers in Europe, Asia, South America, and North America, and Middle East. So these are the emojis behind the faces behind the names behind the emojis. Here are some interesting stats about Minicube. Minicube has more than 685 contributors, which 16 of them have contributed more than 100 commits, like a very active commit developers. You have about 3 million GitHub downloads, which does not include our main method of downloading, which is GCS buckets, but still an amazing number of downloads. You have a 90% CSAT score according to our surveys, and you have had 125 releases in the past five years. 125 times people released Minicube. It's really amazing to me. This is a graph of lines of code of Minicube in the past five years. You have 1.6 million lines of code for this project, which is amazing. That includes the vendor. If you do not count the vendor, it's about 558,000 lines of code for Minicube. And by the way, this method that we chose to count the line of code, it does not include the comments and does not include the docs. It only includes only the go code that we have written for Minicube. An overview of Minicube, something that is unique about Minicube that no other local Kubernetes tool could claim that is a diverse set of technologies that you could use to start a local Kubernetes on your laptop. So whether you are a person who uses Docker, or if you are a person who uses KBM, or if you are using VMware or Parallels, or even if you don't want to use any virtualization technology, you just want a bare metal non-driver. We have all kind of solutions for you. So Minicube is not bound to a specific driver. It's all kind of virtualization technologies that you could have, the most inclusive local Kubernetes experience you could have. And also we have support for different operating systems and different CPU architectures. ARM64 is getting very popular recently. And Minicube is also the only local Kubernetes experience that allows you to set a runtime. I'll talk about the difference of the runtime and a driver in the next slide. But majority of the local Kubernetes out there that you know are similar to Minicube, they only have container D runtime. Minicube allows you to set the runtime. It could be cryo, Docker or container D. Minicube also supports CNIs. You can specify CNI you want. I don't know if you guys saw the security talks on Cillium. Liz Rice had a very amazing talk and she was using Cillium. I was very tempted to replicate her demo using Minicube, and you could possibly get Minicube, you could just easily specify your CNI. By the way, the green ones are the ones that we test them, and the green ones are the ones that we do not have an integration test for them. What is a driver in Minicube? Drivers are this column. Basically, if you want to install Kubernetes, you need a Linux. You need a Linux box. So somebody has to give you a Linux box, either a Docker container or a VM or basically a remote machine like SSH or a bare metal. So that's the driver that you call that a driver in Minicube. Whoever provides you that Linux box that you can install Kubernetes on it. So you can in Minicube start the driver with whatever driver you want. And if you don't specify it, Minicube will automatically select the best driver for you based on your system health and what you have installed. And what is a container runtime? So in this picture, container runtime is these three ones. Container runtimes is the engine that Kubernetes itself uses to manage the life cycle of the containers inside a pod. So this is the container runtime inside Minicube. It has nothing to do with the driver. These two get confused a lot by the people who use Minicube, and I wanted to just in this talk make it clear for everybody so we never have this confusion again. So the driver is the outer Linux box that you get the Linux box and container runtime is the runtime that Kubernetes uses to create pods for you. So the pink one is your driver, and then there will be Minicube's Linux box is the driver. And then inside of that will be container runtime that could be cryo container or Docker. The confusing part is the driver could also be Docker. So you could have a Docker with a Docker runtime, or you could have a Docker container on time. So that's most of the confusing part, but I hope this slide clears the confusions once forever. But here's even more confusing things from Docker. Docker has so many news in the past few months or past year, and Kubernetes has so many news on Docker past year. Recently, in December 8, 2020, Kubernetes says we are duplicating Docker. And people are like, Frick, what does that mean? And so this is about the runtime. It's not a lot the driver. So you could still have the Docker driver with a container runtime. So, but what does that really mean? Does that mean you can no longer have Docker runtime anymore? No, it means that Kubernetes no longer wants to maintain the code that allowed translating Docker shim to a cry interface, CRI interface. However, there's possible to still have container, sorry, Docker runtime. And Minicube chose to continue supporting Docker runtime, not the driver. Why? This graph shows Minicube's main users are local Kubernetes developers, the people who build an image and then deploy it to their cluster. And we have different ways to build an image. For example, in kind or K3D, there is a command called kind image load or K3D image load. Minicube has eight different ways to build an image. One of them is called Docker and. And you could see the Minicube's Docker and is the fastest way to build images on the cluster. The blue one is the Minicube's Docker and and the orange one is the micro Kate and green one is K3D and the yellow one is kind. The lower, the better the number. So Minicube has committed to continue supporting the Docker runtime because it's significantly faster for the local developers. And that's when you build an image, you change iteration on it and you want to build another time and you want to save time on that. So Minicube is five years old. I was in the conference, I was talking to people, I hear a lot of misconceptions and I hear them in misperceptions and I hear them before in the surveys that we collect from the users. And I want to go over the top misperceptions about Minicube. It is true that three years ago, Minicube was very slow. In fact, it was so slow, it was taking about three minutes to get started. We changed that two years ago. We introduced the new drivers, we refactored a lot, we introduced preload. I gave a talk on that in the previous QCon, how we made Minicube faster. But people still think Minicube is slower because when they compare Minicube with others, they compare the VM drivers with, let's say K3D's Docker driver. So if you want to compare these two, you want to compare Minicube's Docker driver with, let's say K3D's Docker driver. And another misconception is Minicube by default waits for the Kubernetes cluster to be healthy. If you say Minicube start by default, we wait for API server to be up and running, etcd to be up and running, and the coordinates to be answering the DNS. If you don't want to wait for that, you can say Minicube wait false and it will be exiting as fast as others. But these are all not very good ways to claim if Minicube is slower or fast. So we built a tool called Time2Kates. And what this tool does is measures how a local Kubernetes tool, regardless of what it performs, and you collect different metrics. So when the tool itself claims I'm done, let's say Minicube start or K3D cluster create, when they claim they are done, that's the first metric we measure. And then when the Kubernetes API server is answering, and then we deploy an app and we measure when the app is actually running on it. And then when the DNS is answering and also we measure the CPU. Past two years Minicube went from two minutes and 35 seconds to 21 seconds. And here's a chart that we generate automatically in GitHub actions per release. So this you could see Minicube against kind and K3D. So we're significantly faster than kind. We are not as fast as K3D. K3D is beating us. And K3D has an advantage. They're not using the SCD. They're using the SQL light for a database. And which is a kind of, you could say, a fork of Kubernetes because the real Kubernetes uses SCD. It doesn't use SQL light. So that significantly makes it faster. So this is the data and the misconception that Minicube is slow. By data, you could prove it that it's a misperception. The second misconception is high CPU usage. It's true. Minicube used to be extremely intensive on CPU three years ago. And we significantly cut down on the CPU usage. Actually, you can check out the talk by Priyawadu improving performance of your Kubernetes cluster in Qcon 2020. She goes over how we did that in a lengthy talk. And we measured the CPU usage again. So the first one is your OS idle. And the second one is Minicube hyperkit driver. And third one is making a virtual box. And you could see Minicube has lowest CPU usage against kind and K3D and Docker desktop. So this is another misperception that it was true three years ago about Minicube, but no longer. Another misconception is like Minicube is not a real Kubernetes cluster. I see this a lot in the surveys. So we say, I wish Minicube was a real Kubernetes cluster, but it really is. We actually passing Kate's conformance test. I put the link to that. We pass a production Kate's conformance test. We do not, we do not recommend using Minicube in production, but we do pass all the conformance test. And also you could actually configure every component of Kubernetes in Minicube. There is a flag called extra config. So you could pass basically any argument to Kubelet, CubeADM, HCD, basically any Kubernetes components out there. You could configure it using Minicube start extra config. So you could actually do any sort of configuration, real world configuration, advanced configuration in your Kubernetes. So don't think of Minicube as a toy. It's actually a real Kubernetes inside a Linux box. Another thing I hear a lot is like I wish Minicube had multi-node. We delivered this feature two years ago. And not many people know about it, but you could actually start Minicube with multi-nodes very easily. You could just say Minicube start-n2 and multi-node. Or if you already have a cluster, you just want to add another node. You could just say Minicube node add. That would be another way to that. Another one that is out there is like Minicube cannot be a multi-cluster. And this is not true as well. You can create a separate cluster with dash P. That's a little bit unfortunate that we chose the name profile five years ago for a cluster. That is the source of the confusion. So if you want to create a second Minicube cluster, we call it unfortunately P, but we have stuck with that because of backward compatibility. You could just say Minicube start-P P1, Minicube start-P P2. And you could actually configure them differently. You could say the second cluster I wanted with container D and the third cluster I wanted with cryo or different drivers and even different Kubernetes version. You could actually say start Minicube with a different Kubernetes version. I actually missed to mention one of the misperceptions, and that is people think you have to install the latest version of Minicube for the latest Kubernetes version. Or if you want an older Kubernetes version, they think they should go install older Minicube version. But that's not true also. You could just do Minicube start dash dash Kubernetes version and pick a very old version to emulate what you have in production. Another misconception is not good for CI. So that motivated me to create a new org called Minicube CI. And this actually is a real world example with a getups model that has example of Minicube in all of these clouds in Proud, Google Cloud Build, GitHub Action, Azure Pipeline, Travis CI, Circle of CI and GitLab. Minicube is really well suited to be used in the CI, especially with the non-driver, which does not need any virtualization at all. It's just straight on the CI machine that you have. So, but if you want to see an example of using Minicube in any of the CI, you could check out this org. Now I want to go over 20 pro tips for Minicube. I will go over them very quickly. And I apologize for speaking fast. And I know it's the last talk of the day. So you find out running out of caffeine everybody, including myself. But I'll go over this 20 tips that I've gathered that I have not actually seen many users use that, but it's possible. You could copy a file into Minicube using Minicube CP command. It's pretty neat. And actually, you can copy between Minicube nodes. The second one is an example of multi-node. You're copying a file called ATXT to the second node of a multi-node cluster. And you say which directory you want to copy that file up to. For more information, you can check out the documentation. There's another way to copy files to Minicube. And this one is very hidden. We don't talk about it that much. And the Minicube home folder is in your home folder.minicube. There is a folder called files. Anything you put there will actually be copied to Minicube on every Minicube start. This one requires a Minicube start. But that's why you have to run Minicube start one more time to get this affected. Another thing I've seen people not use this feature at all is like CNI. They're not even aware of it. And you could try the same Minicube with Cilium CNI using Minicube start. Or if there's a CNI out there that we do not support yet, we support five, six of them. You could pass the YAML file passed to the CNI and it will just install that CNI. And we do the CNI in a specific order that the core DNS will be configured correctly. This is another one, specifying a different container runtime. Minicube start container runtime cryo. One caveat to this, if you want to use cryo, you want to use Minicube Podman Env instead of Minicube Docker Env. Because cryo goes well with podman. And if you want to build an image against cryo, you could use podman. Another one is Minicube add-on list. We have tens of add-ons and then some of them are maintained by Google, some of them are by third parties, some of them by Kubernetes. You can give them a try. But this is an interesting one. Let's say any of these add-ons, you want to use a different image for it. Let's say you don't want to wait for Minicube to bump the Helm's image. You could just say Minicube add-ons enable dash dash images and then provide the image that you want to override to that add-on. This is an interesting tip for myself. There's another trick that you could use. If Minicube has flags called memory and CPU that you could specify the memory and CPU, you could actually pass the word max and it will use the maximum CPU and maximum memory that you could possibly get out of your system. By default, Minicube tries to get a sane amount of memory and CPU based on what your system is. We auto-select it. The better system you have, you'll get the better chunk of it up to a maximum. There's another flag, Minicube dash dash user. I love this flag. We have started writing an audit JSON to Minicube blogs. If you actually run Minicube blogs, you'll see audit table. You'll see what user ran what command on Minicube at what time and how long it took. This dash dash user flag is amazing. I think if you're sharing a cluster with different tools, if you have a few tools that are all sharing the Minicube cluster, you could see who started the cluster, who stopped it, who deleted it, who upgraded the Kubernetes version of it, who applied a different CNI on it, so you can see the flow of the work. If you are sharing a cluster, I strongly suggest adding the dash dash user to all of the commands that you use for Minicube. Minicube starts, stops, anything really. That would be helping with the auditing later. We have a new trick for Minicube blogs. In old times, when something was wrong in Minicube, if you had talked to us in GitHub issues, you would ask you to give us this file, that file, this blog, this log, this one, this one. It was five, six different things you had to paste for us or find for us. No longer is that the case. If you run this command Minicube logs dash dash file and whatever file you want, it will collect all the different logs from different places, from different inside and outside Minicube, and it will put it all in one file. This is an interesting way to create a GitHub issue. There are other pro tips that I want you guys to try. If you want to be a pro Minicube user, there's eight ways of building images with Minicube, and it's on our website. So give it a try. There is another tool that I think many people do not know about it. It's called Minicube Cube CTL command. So if you have not installed Cube CTL, you could actually use Minicube's Cube CTL. You can say Minicube Cube CTL dash dash gets parts. You do not have to install Cube CTL. But there's another trick that I was amazed by the contributor who contributed to Minicube. I was like, oh, this is pretty cool. If you actually rename the Minicube's binary and call it Cube CTL, the Minicube binary will no longer act like Minicube binary. It will act like Cube CTL. So this is an interesting thing, because you do not need to download both Minicube and Cube CTL anymore. You can just download Minicube, rename it, call it Cube CTL. It will be Cube CTL. So another trick, another pro tip. Another pro feature. I'm curious, anybody ever used Minicube PaaS? A show of hands? Nobody? Okay. This is a very cool feature that nobody has used. So you can actually, when we were doing a lot of analysis of what taking Minicube's CPU usage, API server of Kubernetes is a beast. It takes CPU. And do you need Kubernetes API server if you're a local developer? Do you? You need it sometimes. You need it when you apply your .yaml file, but after that you don't need it. Your app is running. If you pause the Kubernetes API server, your app will still be running in the PaaS. So PaaS Minicube, whenever you don't want it, it will pause only the Kubernetes API server. Your app will be still running and it will save a lot of battery if you're in a coffee shop, you don't have a charger. I mean, if you pause, but still your app will be running inside the Kubernetes cluster. We even have something better called AutopaS. In the previous one, if you paused it and you want to apply another app to your Kubernetes cluster, you had to do Minicube on PaaS. What if Minicube would do that for you? Like each time you apply a new .yaml file, it will unpause itself. And each time that you're not using the cluster, it will pause itself. You have an add-on called AutopaS. This is a dummy diagram that I draw really. I'm not very proud of what I draw. Sorry, but this is how it works. We have a transparent reverse proxy that takes care of AutopaS in Minicube. But if you want to give this a try, Minicube add-ons enable AutopaS, a pro tip. Another pro tip, we have a new driver that I'm pretty sure nobody uses. SSH driver. Anybody use SSH driver? Okay, half of the hands went up, but it went down immediately. Okay. So this is a way you can install Kubernetes on a remote SSH. Or if you have access to a remote machine, you can install Minicube on that remote machine for me. All you have to do is just give us the IP. You'll go SSH into that. Of course, you should have SSH access to it. And then we'll just strap a Kubernetes inside there for you. Another pro tip, you can SSH to the Minicube cluster itself using Minicube SSH command. And if you want to only run one command and be done with it, not go inside. Minicube SSH dash dash, whatever the command it is. Let's say pwd. This is another one. You would be surprised that this is one of the features that people asked us to implement. And this is called schedule stop. What does it do? If you say Minicube stop, schedule five minutes, Minicube will stop itself in five minutes. Why people asked for this? Well, people use Minicube embedded users like in IDEs. So some people say close their ID, but they don't want to wait for it. They force kill it. And that don't give the ID to stop Minicube. It's enough time for a stop Minicube. So this basically says, if I don't hear anything from you, I'll stop myself in five minutes. So in that use case, the IDE was every three minutes was scheduling another Minicube stop for five minutes. So it's basically refreshing it. So this is a way to schedule Minicube to stop. Another new feature of Minicube that people probably don't know about it is output JSON. So you can actually make Minicube talk to you in JSON instead of emojis and texts. And this is also used for embedded use. The format for JSON is actually the standard CNC of cloud events formats. The mount command, if you want to be a pro user, try the mount command. What is mount command good for? Mount command is good for if you want to have persistent data outside Minicube. So if you want to delete your Minicube cluster, but you want to keep the pods data somewhere on your own machine, you can mount a folder from your Mac or your Windows into the pods. And then you will have real persistent data. This is a pretty nifty feature, but I have not seen people use it that much. Because I don't see that. Sometimes I see GitHub issues on them, but corporate search. This is an important one because I get a lot. So a lot of us work for corporations and corporations have their search, right? Who here has corporate search in their company? Yeah, a lot of us. Yeah, and corporate search are very annoying because you want to curl Google. It does not curl Google because I need the search. So how do you add your corporate search to Minicube search? So basically copy it to this folder and Minicube will take care of it. Basically, we want to copy it to this folder and then run Minicube search one more time. So Minicube copies those search into the chain of the search and Kubernetes can talk to the outside world using your corporate search. Another tip. You have two more tips left and I am running out of time. GCP off is a new add-on that we have. You know, in old times when you had to, let's say your laptop is authenticated to GCP and your pod wants to be authenticated to GCP. You had to create the secret, apply the secret. It was such a pain. So we developed this add-on called GCP off that automatically gets the credentials from your laptop and puts it to every pod inside your cluster. So it will automatically have access to GCP just like your own laptop has. There is another pro tip. If you do not want a specific pod to get that secret, just apply this label to it. GCP off keeps secret and it will not get the secret. And there is a link for more information on that. Another pro tip. We have two more left. Delete on failure. So Minicube philosophy from the beginning has been be backward compatible and do not delete people's data. So if you start Minicube and we crash halfway, a lot of the times deleting Minicube and starting it again just works. Sometimes it's just a virtual stitch in hyperbill gets stuck. Something gets stuck and you're just deleting it works. If you want Minicube, allow Minicube to delete on itself and try again. You can just say Minicube delete on failure and then Minicube delete itself and try again on a failure. But by default, we do not do that because you want to respect your cluster if you don't want to delete it. If maybe you have something interesting in there. Try our load balancer is another pro tip. Minicube I think is one of the only local Kubernetes that has direct IP access to the load balancer in the VM drivers, not in Docker drivers. So Docker has its own limitation. And very many more, we have three interesting links Minicube tutorials Minicube handbook and Minicube FAQ check them out. Now back to Docker. Docker has had so much news past year. Okay. The recent news is they are going to be a paid tool for non Linux operating systems. And but they will be free and open for the Linux and open source users. And there's some blog posts on that in read. This is a took a screenshot of the Minicube surveys before I come to the coupon. So it's like why you use Minicube is a Docker desktop going away looking for dust of Docker desktop alternative because I didn't want to be tied to Docker desktop bundle specially they were forcing updates. I need OSS alternative for Docker desktop as a replacement for Docker desktop. This is like people are flying us with messages on, can you be a replacement for Docker desktop? I mean, we are, we are a local development Kubernetes tool we are not meant for that, but we can be. So this is how there are two ways to use Minicube as a Docker desktop replacement. One is using the Docker and method. What does that mean? You can install Docker CLI you can using brew or whatever, but do not install the Docker desktop or quit Docker desktop or exit it. And then use the Minicube Docker and if you evolve the Minicube Docker and your terminal will be pointing to the Docker inside Minicube and Minicube is a Linux box which means it's always going to be free as Docker claims. So, and then your Docker CLI if these do Docker PS or Docker belt is actually talking to the Minicubes Docker is no longer talking to the Docker desktop, or there's no Docker desktop in this case. So you could easily actually replace Docker desktop with Minicube. There's one method one. There's a second method. We introduced a new command in Minicube called Minicube image build Minicube image, which has many, many sub commands like image, build image load image delete image list. There are many sub commands for a Minicube image command, but you could basically build your images using Minicube without Docker at all. You do not even need to have Docker CLI installed. You could just say Minicube image build dash T and then pass to your Docker file. Basically the same syntax with Docker build. And underneath we are using built kits. And so your image will be inside Minicube. This is like a more Minicube native method. The other one is like you're pointing to the mean, the Docker inside Minicube, which either way it works. So there are two methods that a lot of people ask and I hope in this conference, I answered all of these users questions with these two methods. And if I didn't answer it correctly, hit me up on Slack. I'll make a better or more clear instruction. There are some lessons I've learned in Minicube in the past three years of being a maintainer. We've worked with hundreds of contributors. We have had 675 contributors and many of them are very active. There was a time that I was looking at the metrics. There are more new contributors to Minicube than Kubernetes and they're more active developers to Minicube than Kubernetes itself. And this was based on the culture that people before me built it and I tried to preserve it. Thomas Strumberg did a great job in doing this and I followed his lead. Like being inclusive in action. Minicube tries to be as inclusive as possible. We try to have support different languages and we try to have our GitHub issues welcoming so people can talk to us. And we are in a very unique place that a lot of people who try Kubernetes, they come to Minicube for the first time. Some of them are not very technical, but they are very interested to be technical. So it's like a very interesting energy. Another thing that I learned in this past few years was recognizing the contributors and we developed actually tools to recognize the people who do any sort of contribution. It didn't try helping in GitHub issues, responding that. And then I'll show you an example of that. And then the very one is like scheduling short one on one meetings with the contributors helped a lot in energizing these amazing Kubernetes. And also inviting to the Minicube exclusive meetings. We have two Minicube exclusive meetings. It's inviting you guys to that has been a big part of our community management. Here is one of the tools that we developed called pull sheets. Basically, each time you release Minicube in the release notes, we put a link to the charts and on who did what sort of help in what category in Minicube. And if you're not a technical person, but you want to answer people's questions, you get your name recognized in this leaderboard is going to be on Minicube website forever. So we try to recognize contributions. Another maintainer lessons that I have learned is embracing the human part of the softwares. This slide was very important for me and I wish my talk was title differences I could have dedicated the whole talk to this. I'm so passionate about this one, but I could only fit it in one slide and throw it very quickly. What I've learned that some of the stereotypes about software engineers might be not too wrong. There's some, our profession attracts different type of personality and very neurodiverse personalities people with OCPD ADHD cluster C and working with these people learning them on unleashes the power of maintaining a big project. If you learn that some people get obsessed on something and they cannot let it go. That person could be amazing if you want order in 1.5 million lines of code. But you also want to learn what is that strong part of this person, what is the weakness of this person or another type of person. He might miss out paying attention to the biggest thing, but has the most creative mind, and he also need that type of person. Or you have the cluster C type of person who worries all the time. If you want to implement anything, worries all the time about the security implications of this. And the art of this is how we balance all of this. What is the budget that we have? What is the budget for security? What is the budget for creativity and what is the budget for order? It takes, I wish I could have more time to talk about this. There are other talks in KubeCon that they touched this topic a lot, but I really learned a lot in the past three years. Learning the humans around you goes a long way to maintain a healthy project. Once you learn your humans, once you learn your contributors, you know what type of health they might need. Some need assurance, some need keeping on track, some need getting started. Another lesson that I learned is don't take anyone for granted. Some people come to these open source projects to take a break from their day jobs. The worst thing you could do to them is like take them for granted that you're, yeah, you're supposed to do this. Not taking anyone for granted also is a lesson that I've learned. And also another lesson is do not burn out. This is another lesson that I have seen a lot of people in open source. They come work really hard for two years and they burn out. There is actually a good talk in this talk, this KubeCon schedule. I recommend watching it on YouTube. Check out a vulnerable tail up burnout by Julia Simon. This is an important lesson that I learned in the past three years of being in Minikube. Another lesson that as a Minikube maintainer I learned was like learning when to say no. And that was very essential for this old, big project to keep going. One is the spam contributions. I don't know if you guys have seen any spam contributions. People literally write bots to make contributions to projects. Another thing to say no to is a promotion-driven development. There is an amazing Twitter discussion, promotion-driven development. People write codes just to get promoted and you could just smell it. And it ruins a project's future. And I have learned to say no to that. Another thing to say no to that is hard to maintain code. Those who are harder to maintain code does amazing things. It makes the project so complex that only that person could maintain that. Of course, do not emerge or accept any contribution you do not fully understand. There are some special interest people. I have seen that people with some special interest come to your project. They want to try to get this into their project. That's another place to say no. Another place to say no is potential security liability. I have seen a few times people wanted me to add a future that I saw potential security vulnerability. This is a third-party image in this code who can guarantee the next five years this image is going to have the same integrity as today. This project has been five years. This is another place to say hard no even though it adds a feature. And also another place to add hard no is when somebody adds a new feature without integration test. Hard no. This is amazing lessons I have learned by mistakes or by others. Another thing that I have learned is guarding against your own bias. Unconscious bias is unconscious. It means you don't know you have it. And there is at least seven of them that I strongly recommend. Take a Wikipedia page and read about it and put that in your mindset. Another main theory lesson is GitHub issues is an amazing source of information for future work and improvements. When I look at the GitHub issues, I see that the place that I learned from the users and when I am doing PR reviews is where the users learn from me. So they're two-way communications. But GitHub issue is where I am supposed to learn from the users. And for that reason, we actually developed a tool to triage the GitHub issues. This is called triage party. It's a multiplayer game that actually a few other CNCF projects are using it. And we started at Minikube, the other projects are doing it. And basically this helps you to triage the issues more effectively using a crowd sourcing technique. And there's different categories of triaging. For example, let's say somebody creates an issue in Minikube and say, oh, there's this problem with this and I asked them for more information. And then two weeks later, they answer with more information. But that gets lost in 500 issues in Minikube repo. Wouldn't it be nice that a tool would tell me that the person who you asked for more information now has more information. And that would be in a different category. That's one thing that we did in Minikube. Minikube, another interesting that we did is we translated Minikube to multiple languages and Minikube now speaks your language. So if you want to add your own language to Minikube, it's super easy. It's a JSON file. Just translate the JSON to your own language. Currently, we have French, German, Spanish, Chinese, French and Japanese and Korean and Polish. Another thing I like to touch in this talk is the Minikube integration test. There's a challenge in Minikube integration test. And the challenge it is we support many, many platforms and many, many runtimes and many, many drivers. And we also have legacy features that we do not want to break. Five years of user-based features that if we break them, we break somebody. So those are the challenges. And we have different types of testing. Unit testing, integration testing, Kubernetes performance test, pressure test and translation test. Currently, I do not know any other projects in the CNCF world that has as diverse testing environment as Minikube. We get tested in GCP, Azure, AWS for arms 64, Equinex, Bear Metal, Mac Studio, which are basically physical Mac machines and GitHub actions. And also we have Alibaba in progress. And I would say this is the most powerful part of Minikube that we get tested in many, many locations and in many different drivers and runtime combinations. Another big challenge for Minikube was flaky tests. We have many, many tests, hundreds of tests, and in a PR you would see a failure and the failure would say this test is failing. So how would I know if this is a real failure that is caused from this PR, or how do I know if this is from a flake test? So we actually built a tool that measures the flakiness of a test and comments on the PR and tells you the flakiness. And that's also in our website. You could check it out, the charts for Minikube flake based. And this is based on a tool called Gopuk. Gopuk, we built a tool for Golang that converts the integration test to HTML. But this has helped us to make the PR reviews much faster. You can see a test is a flake or not very quickly. Another thing that we have done in Minikube, we have created a Minikube bot that makes a lot of the PRs for us that we used to do by ourselves. Like bumping the Golang, bumping the Kubernetes version, bumping this, bumping that. All of that is done by a robot now. We don't have too much time. I think we actually ran out of time, I believe, and I'm very amazed that you guys stayed because I didn't pay attention that we ran out of time. But if you want, you can follow Minikube's Twitter. We have a Minikube bot called Minikube dev that automatically pushes updates to Twitter whenever there's a new Minikube release. So follow Minikube bots. And for future, we're going to have VM driver for ARM64. We have ARM64 support for Docker driver, but not for VM driver yet. And we also have our plans for maybe unifying the ISO and the container based image. So with that, thank you very much for staying. And I'll stay for if you have questions, I'll be more than happy. And sorry for running out of time.