 Welcome everyone for yet another presentation on KubeCon and CloudNativeCon. Hope that you have really good conference so far and you've been able to see some really awesome presentation. We are really happy to be here and share our story with you. So together with Vano, we wanted to present today a few of our business cases about open source and how the Dutch government benefits from using it. Yes, as the title already says, water is the driving force of innovation. We will look at some major projects related to water in this presentation. And for now, let us introduce ourselves. My name is Tom Klimek. I work at GrapeUp and I work with RWS, Rikswaterstad for already several years. And I'm a delivery director and currently I'm focused on delivering CloudNative solutions. And my name is Olma Brouwer. I'm part of the CloudNative application platform team. We basically operate and manage our CloudNative platforms in Rikswaterstad. Rikswaterstad, well, who are we? What do we do? We are the executive agency of the Dutch Ministry of Infrastructure and Water Management. We basically promote the safety, mobility and the quality of life in the Netherlands. Well, what's safety about? That's about protecting us against a flood and high waters. Protect mobility is related to the highways and the waterways we have throughout the Netherlands to manage people moving about. And the quality of life is referring to the environment, natural environment, keep Netherlands green and energy neutral and also quality of drinking water. We have quite a few employees and we have been around for a while. And I guess you can see a lot of pictures of water and Rikswaterstad is basically the executive agency for the water management in short words. So I guess we need to talk a while about why the Netherlands needs to put so much focus on the water management. So it's really different perspective. I'm not from Netherlands, I'm from Poland. So I can give you my opinion about it. So usually if you are transporting goods or you want to travel, you have your highways, you have your local driveways. So you just go into your car, put some cargo on the road track and that's it. There are ships traveling between countries on the sea and so on. But the Netherlands have really different perspective. There is very rich network of the water canals and they are widely used for transporting goods and people around the country. And this is pretty interesting story and it all has started because 26% of the Dutch territories are under the sea level. So that means that they need to be drained in order to be used. So as far as this story has been starting several centuries ago, they were the Netherlands, the Dutch people were building a special water canals just to drop the water from those territories into the sea. And once those canals were ready, they were using that from other purposes, first traveling and then transportation and here we are. So on the picture on the right hand, you can see a water canal going above the highway. It's a pretty awesome picture and you can see how those two transportation types can work with each other. And on the other side, you can see a special device to monitor the water level in the most important canals in the Netherlands. So that gives you some kind of idea that this is pretty different mindset and it's pretty other things to worry about on daily basis. So what is our mission? There are pretty important mission critical systems that we need to operate. There are a few teams that are the different parts of this process. So our goal is to deliver the best tools and technologies that we can use in order to have the most resilience environments and being able to operate in the worst case scenarios. Worst case scenarios, like for example, getting our feet wet, we prefer to keep them dry. So that's one of the most important reasons we are around. Yeah, that is really nice way to put it. So is it stressful? John from our team doesn't complain. He's only 26, he's a rookie, so good know, maybe he's lying, but yeah, it is stressful. We aim to have our technologies to have it less stressful. So let's talk technology for a bit. So let us dive into our architecture. Yes, let's talk technology. We basically have drawn three pillars here. On the left hand side, we have Borsh, which is an open source VM orchestrator, basically managing VM deployments. And we use Borsh to deploy automation, tooling like concourse, monitoring, like Grafana and Pomicius. We use Minio for S3 compatible storage. We have Shield for making backups. And since our standard workplaces don't have all the tooling on board, we would like to have. We may also manage a set of jump boxes, which we control and on which we can install all the tooling like a kubectl CLI for our needs and also our customer's needs. On the right hand side, we also have the platform we started out with, Cloud Foundry, which is a past platform, platform as a service. We have quite a few applications running there. And also our major data services are provided by the platform like Rebitem Q for messaging and Redis for database usage or session management. However, we are at KubeCon, Cloud NativeCon. So let's focus on the central pillar here, which is all about Kubernetes, of course. And we have also related services like Harbor for container management. Yeah, definitely. So right here on the screen, you can see a simplified architecture of Cloud Booster platform, which is delivering the Kubernetes into our operations. So going piece by piece, you can see a orchestration layer at the bottom, the ops automation. This is the layer where we have Kubernetes delivered by Kube Spray. We use Terraform and Ansible in order to automate deployments, also Helm to deploy other stuff. We have a little arrow to backup and restore Kiko and vault in order to do user authentication. The heart of the platform is, of course, the Kubernetes clusters. We deploy them for our customers with the different tool set. And for that, we use the marketplace. We also have the telemetry tool sets based on the most popular Prometheus Grafana and Elk, which is delivered by OpenDistro. And we also have the workload management layer, which is helping with the easy delivery. So we use their hardware, Clare Traffic and Nginx as an optional interest controller, ingression controllers. And I guess you can see a lot of logos here. Most of them, if not all, are provided by the CNCF community. They are all the cloud native landscape. So I guess we've come to the right place and we wanted to show you that technologies that we actually use in our daily operations. But enough about the technologies, let's dive into how our customers are actually able to benefit from them. Yes, when you look at open source, there are several benefits. Here we listed what we think are the major ones. First of all, open source. We can have, of course, support through the open source community. And not only the community, there is of course also vendors out there, like you, for example, who can help us manage it all because it's quite a lot of technology, quite complex, lot of story choices, and it helps to have a vendor helping out. That is very true. Another important point is starting immediately. And this is especially important for the enterprise-grade companies. If you want to use open source, the process of using that building MVP or POC, it's a lot quicker. The procurement process, everything is in place right away. So also the community that Ono has mentioned, it's really helpful to start immediately and have an answer to your question. Will I benefit from that? Another benefit is we don't need any paid licenses. We don't need any upfront investments. We can basically download the code and see if it fits our purposes. An example, a while ago, we started developing a Postgres broker. So basically, we could download a Postgres operator, install it on Kubernetes and see if our broker could communicate. Another very nice benefit is transparency of the solution. And this is a good way to show how open source has turned around the cards. 15 years ago, we wouldn't have that conversation and we wouldn't have that presentation. Currently, the transparency, you can call it the two-sided sort that transparency is good and bad, but currently it's rather a good thing because if there is any vulnerability in the code, probably someone else has already found it. There are really major companies supporting the community because they also want to use it on a production in a very demanding environment. So it's a really important point nowadays. And last but not least, no vendor login. Even though we use a lot of open source, like I mentioned in the beginning, it's nice to have a vendor helping us out. On the other hand, we don't really want to be tied to a particular vendor. And because it's all open source, the basis is all the same for everyone. So that gives us the flexibility to switch if and when we decide to do so. And I guess I can say all of those benefits are also risks at some point. So another risk to consider is if you want to use the open source, you really need to know it. So for every requirement, there is at least five tools that you can use. So that's why we also wanted to show the Code Native Computing Foundation landscape. It's the least of all of the technologies you probably know. It's not all of them are open source, but still even looking at the CI-CD tools that you can use, there is at least 10 that are open source. And for the most common use cases, every each of them can deliver the same experience. So we really need to know what is your exact requirement? What are the five of the next steps that you need to take and which tool to use? Yes, as you mentioned already, there's a lot of choice in there. And by the way, the great outboxes are the closed source options, right? The white boxes are the open source ones. Exactly, yes. So here we are. We've talked a lot about the technologies. So maybe let's reach into our main focus and business. So right here we have prepared some mini comics to show you different perspectives. So I will give you a few seconds to read it out. Hope that you have already taken care of that. As you can see on the picture, using technology is not always the best idea if you don't know how to use it. So it's always good to know how you can benefit from the technology that you're picking. And that's exactly our point from the previous slide and showing the CNCF landscape. So moving on, let us talk a bit about the business and how do we benefit from those open source technologies. Yes, let's talk about business. Our first topic here is about the National Water Monitoring Network, which is one of our major customers of our platforms. Basically what you see here is a map of the Netherlands showing the sensor locations. Note that we have sensors not only in the Netherlands itself, but also in areas around it, mainly the North Sea and the upstream rivers. Basically we take measures of water quantity like water levels and the speed of water is moving about the water quality, think of things like salination levels or pollution, weather information, whether the wind is blowing, et cetera. And last but not least, also the state of our objects, whether bridges and gates are open or closed. Scope for this all is to basically smart water management during drafts in high water. And we also use this input for shipping reporting. And of course, last but not least, it's used as input for our flood defenses. Looking at this picture, it seems that the whole Netherlands are like IoT country, looks pretty nice. And I guess for me, it's really nice to see that those receptors are not only in the Netherlands, but also in the countries around the Netherlands, because you can have early feedback if there is a flood in different areas that can impact you. So that really is good planning. Here we have an overview of basically what we call the generic Rijkswadstad IoT procurement chain. Starting on the left, we have the sensors providing the input. The sensors are connected to our standard IoT gateways, which are basically RaspberryPars running SUSE Linux with Docker on top to collect the data and to optionally validate and process the data. Then we have a link towards our data centers. We currently use two for these platforms, and basically we ingest the data into the Active Data Center. We should note here, this picture shows only one setup, but we have everything duplicated and we explicitly switch between data centers at least twice a year. So we know that both data centers are always available and fully functional. In the centralized part, we have some apps running on Cloud Foundry and also a major portion of the processing is our RabbitMQ process, which is basically handling all the messaging. And on Kubernetes, we have the event screen processing, which is doing all the business logic, the AI, the processing of data and storing it into our dream plan, which is like a massive parallel processing Postgres database. At the right-hand side, we also, of course, not only store and process data, but we also make it available to all relying parties and for the distribution we use ESB. So maybe not everything is open source, but that's a good match. The best of both worlds. Yeah, exactly, exactly. So our next system that we wanted to show is the vessel management that is operated in the real time. So this is basically a system which calculates the perfect and the best way for all of the ships to travel around the Netherlands. So it is using not only the data from the ships and their position, their speed, but also taking into consideration all of the things around. So traffic poles, logs and bridges. So this sounds really complex and it operates in the whole Netherlands and also gives feedback to other systems in the countries around the Netherlands. So this sounds like very big planning job to process. So that's also another good example how to benefit from using open source technologies and help yourself. Yes, you not only have traffic jams on highways, but we also might have traffic jams on waterways unless we take measures like these. Here we have, well, this is really actually already live. This is our CI CD, which is part of the IVS Next project, our shipping management project. Basically what you see here is on the left-hand side, we have the continuous integration which is providing the input, new versus of microservices. Our Go CD is used for managing the entire CI CD environment. Images are stored into Harbor and made available and configuration files used by Helm, Helm charts and the like are stored in GitLab. When these are combined, you end up with images you can deploy in communities. And first, of course, we would put it into tests and then we, of course, we want to make sure everything works. So we run some automated regression testing. When that all passes, we can deploy it into the acceptance environment. And then there is the SCORM meeting where we discuss or the sprint meeting, I should say, where they discuss the results. Once that is improved, we can move towards pre-production, run another automated tests, full-chain test and once that all passes, we can put it into production. Note that all the software as you mentioned here, it's all open source, which is a nice combination of and the power of when you start combining tools. And another case that we wanted to show is also pretty nice and I like it very much because this is an application that everyone in the Netherlands can use. It's a different situation than two previous applications which are running the mission critical systems to help the internals Dutch services in order to keep the Dutch feet dry. So this one is also doing that part, but it's public application. On the left-hand side, you can see the UI of that application is very simple. You can put your zip code, your address and check your current water level. And if there is a flood, if you are impacted by that and how to prepare. So I guess it's another part of seeing a different mindset in the Netherlands. If you want to buy new TV, then you can just see the current water model. And if it's a good idea to put it on their first floor or the second floor, it's really different perspective. And I also should mention that in the back-end of this application, it's really complex water model computation. So as far as the, it looks pretty simple. At the back-end, it's really complex and we have a lot of microservices doing that work. And it's also very nice to mention that the peak performance of that application would be during the flood. So when it's actually needed the most. And I guess I want to say something about the last usage that was unfortunately needed. Yes, unfortunately it was needed but the platform met the challenge. Normally it was running in a single container but we saw it go up to 16 containers to handle the additional load. So that was a nice example what automation and platforms can bring. Yeah, that's an unfortunate event but a very nice scaling scenario that we had. Okay, so after presenting those business cases, we also wanted to share with you our lessons learned. And the first one for sure is the quick communication and feedback loop during operating that big projects, big operations like mission critical systems. You will have a lot of teams. You will need to coordinate the work between them. So it's really important to have a quick communication and a very good feedback loop just to stay on top of any issues and work together on any problems that we can have. Yes, next to the, of course, communication which is always important, but we can't stress it enough. Having the right mindset, basically gain knowledge be greedy, platforms are complex. When you look at platforms like Kubernetes there's a lot of technologies, a lot of moving parts. It can overwhelm you quite quickly. So you need to learn, be mindful, keep learning. So to do that, we have of course some tools we use. We have like an internal wiki where we document all the things and all the specials about our platform. We also organize internal pizza sessions to tackle a certain topic. And of course attend events like this one to get new ideas. On the other hand, having the right mindset is not only about gaining knowledge also about sharing it, be generous. Not only within the team, but also external to the team. We have a customer feedback loop. We meet regularly with our customers. What are your requirements? What are your ideas? We can provide them the information what we are planning. We have onboarding sessions to help customers find the right technology or the right tool for their requirements. And we also organize workshops and internal and external presentations like this one, for example, to bring out the good message. Exactly. That's very nice thing to do. And the other part of the mindset is that you need to all of the teams, you need to have them look at the same picture who have the same approach. So it's always important to share and spread the mindset. During running those peak operations, you will have different external teams, probably will be around delivering the platform or the software delivery. You will have different teams also working on the other level, not only software, but also maintenance of some machinery and devices. So it's really important to spread that mindset, see the same approach and work on the same problems in the same approach. Apart from spreading right mindsets, of course, you need visibility. What is the platform doing? What are the applications doing? Monitoring and alerting is of course vital for good operation of your platforms. Many things can happen. Many things can go wrong. Monitoring can help you. You can watch a disk running out of space. You can use trend analysis. Commitures can give you nice warnings up front like, hey, this disk might fail up in the future. Maybe you should take action. That's pretty cool technology and very useful to monitor the platform. However, there are still some dangers. It's very easy to fall into the trap of, okay, we've got it all in control. We manage everything, monitor everything. Prime example, for example, we had Bosch mentioned in an earlier slide, managing and deploying VMs. We had agents running all those VMs, monitoring all data like CPU, memory, et cetera. Everything was monitored except for one VM, the Bosch director itself. It was not part of the monitoring. So of course, Murphy's law strikes and we ran out of disk space and we had to fix things. And then we decided, okay, we have to fix a blind spot. Next to monitoring, of course, one of the main activities we deploy whenever we come across something we have to do twice. We think, ah, maybe we should automate this. We have a pretty small team. We have to manage platforms running hundreds of applications. So to keep it manageable, we try to automate wherever we can. We want to use a version control. We don't want to get automatically the latest and greatest. We want to keep things in control. We want to make sure the business is repeatable, predictable. And to keep everything up to date and also make sure environments are consistent. We also use automation, not only for platform updates, but also for things like automated reporting. It's always important to keep an eye on things. Even if you don't touch anything, things are always moving. Time has also changed. Certificates, for example, might expire or passwords might expire. You want to keep a watchful eye for that as well. Related to that, we have our smoke tests. That's an idea which we don't really see a lot of people doing, but to us it was like a lifesaver. We run automated smoke test after a few minutes, check if the platform is still operational, can we deploy applications and also are all our backing services like RebinQ or Postpress are they all still available and reachable? Basically a proactive monitoring of the platform that saved us many times. Spot issues before our customers get affected. And the nice thing about it is that in the office in Delft, where our team is located, there is a huge screen on the wall. Big screen, yes. Yeah, the whole result is visible to everyone in the team if you are working from the office, of course. We just need to add the special, let's say, animation that if anything goes wrong, we should have this fire on the wall and some siren going. That would be pretty nice. So the last thing for today to mention, because we can go on and on by the lessons that we've learned but the last important thing that we would like to highlight is focus on the big picture. So definitely we'll have several teams working in the big projects, big operations and we need to have like a shared goal and we need to focus on the bigger picture what do we want to achieve and to not look at the small part and just limit ourselves. So this is also pretty important. So I guess while looking at all of those lessons learned, some of them that we didn't put them here as well, when we were trying to prepare that side with Ono, it's just come across to our minds that this is basically yet another example of the DevOps transformation that we did. So it's really nice to see that a lot of companies are working in the same mindset. You really need to be working on the same problems together. There shouldn't be any high losses in the companies. There should be a very good plan to release and so on. Yes, and also that applies to multiple levels. It applies to us as a platform team, but also to our developers who actually manage the applications. And with that picture, we wanted to close our today's presentation. Thank you very much for your time. Thank you all. I hope you enjoyed our presentation. Now we will... So, do we have any questions? Anyone in the audience?