 Let's start. OK. Hi, guys. This is Ivan Kraviko. He is a solution architect, and he is working for Mirantis Company. And Ksenia Pakofya is a product manager from Wargaming. So together we are going to tell you a little bit about our cloud journey, what the Wargaming company is, what was the cloud use case. Before we started that journey, how we did, how we proceed the whole way, what did we learn to achieve while doing that project, and what are the next steps. So, Ksenia, please tell us a little bit about the company. Yes. Please, before we start, how many of you play computer games? Rise up. Thank you. Great, girls. And who prefer console games? Just two? OK. So we have a dry hardwood trick here. Nice, great. And which games do you prefer? Maybe Warcraft? Oh, nice. Dota GTA, GTA, Call of Duty and Fallout. Great. And do you know about game World of Tanks? Great. I thought you have a right audience here. After our presentation you could find me and maybe I could help you with some pre-gold, OK? And I'm from Wargaming. And Wargaming is a global gaming developer and publisher. You can see all figures on the slide. And the most important thing to know is that we have a big team, we create cool games, and we have lots of users worldwide. We deliver legendary games globally with passion. As you probably know, the most famous game is World of Tanks. On the 12th of April, this game project celebrated fifth anniversary since it released in European and North American regions. G-Core, G-Core has been a long-term Wargaming partner. Supplying our IT infrastructure, of course. Our cooperation with G-Core began with a shift to online gaming with the World of Tanks release. We choose G-Core because the company was able to meet our current service needs and well as supply a solid foundation for future growth and our global presence. As you can see on the slide, the process of infrastructure allocation are really complicated. And the technology stack is huge and mostly based on open-source components. Where there are four main steps to the product go from developers to a gamer. Staging trunk, sorry. This is the development and the QA environment. They request the personal, personalized environments to test and debug the code. Debug the code, sorry. Sting. Staging stable. While preparing to the new release, DevOps team requests and configures new virtual environments to deploy and execute a set of automatic tests against the application. The third is a production test. Included super test and common test. It's real players, but just the focus groups. All environments required by development engineers. And it's like the close public beta testing. And production DevOps teams require environment and configures new virtual environment. Work team interaction is based on Jira issues. In order to get a new virtual machine, it's necessary to create two different issues. And it usually takes three or more work days in total to resolve it. This is a too long and difficult way, including the human factor and different errors. Some of our services were still running on bare metal hardware and added even more complexity. Why cloud? For wargaming, it gives us opportunity to find a better service to our players. With the possibility to quickly increase or decrease server capacity in different regions or during different time of the day. We're able to fulfill the demands of fluctuating requirements. It also allows us to better structure our costs. We pay for the capacity we use, not the service we rent. Wargaming has engaged Mirantis to implement a cloud solution based on the Mirantis OpenStack. Fairly. To ease our current pay points and offer a flexible solution to our online presence team cost effectively and without compromising on performance and scale. There are two business goals which are most important for us. The first is to reduce the cost with Mirantis OpenStack and increasing infrastructure utilization, of course. And the second is to reduce time to production by using API and automation for manual operations. For use cases, we have taken as a basis a four steps workload lifecycle, staging trunk, staging stable production test and production. Last two phases imply working with real players and real users. Now, even tell us how the Mirantis team has helped us. Thanks, Xenia. I personally also play World of Tanks and World of Warplanes from time to time. So it was quite an interesting experience for me to work with the team and have a quick glance into how do these games actually being developed. Well, from Mirantis OpenStack perspective, this project was a good sample of how we see OpenStack process should be done. So we have all the main phases here. First, we assessed what are the actual goals. We prepared the content in the modular and repeatable way, rolled out the clouds, and now are helping the team to operate it and plan the next steps. Let me dig deeper into each of these phases. So at the very beginning of the project, frankly speaking, the requirements were kind of unclear. So instead of just answering the initial technical requirements, we empowered the Xenia team with our architects, sat down and started thinking, what are the actual business cases? What are the detailed technical requirements behind it? So after it was in a formal workshop, we collected the priorities that gathered all the requirements and helped the guys to understand and map it to the actual OpenStack capabilities. Thus, we looked down the MVP and proceeded further, preparing a number of documents. You can see some diagrams from these documents on the slide. Going all the way deep down from the use cases through the overall project architecture and the deployment architecture down to actual wrecking diagrams and networking diagrams, both for MVP and for some suggestions how to do the next steps afterwards. When the project scope was agreed and finalized after this activity, we proceeded to develop and phase. As it was based on meridians OpenStack, it was using few. It allowed us to do all the customizations there was a number of in a modular way before the actual deployment. So we started out with detailing what are the exact requirements for each of the plugins to be developed, put out our own lab lab environment, written down the code, prepared the QA processes and policies for that, and also prepared all the quality assurance assets needed to hand over the solution after it's rollout. Actually, four main field plugins were developed in this phase. One of them was Active Directory Integration for Authentication back-end. Our work gaming team is mainly using NFS for their storage back-end. So we also prepared a plugin to use NFS, both for fmrl for volume and for images. For volumes, we used RemoteFS as a center driver here. There also was a need to disable end-to-spoofing rules. As some of the VMs are used as a host for containers. Thus, multiple IPs per VM were needed. So we also made a plugin to enable end-to-spoofing rules both in Nova and Neutron in this case. And the last one was the integration with External Puppet Master. So the Puppet Server built into Fuel was used for initial deployment and afterwards the environment was integrated with the configuration management system for the rest of its lifecycle on work gaming premises. When all the content was ready and we agreed on the test plans on demo scenarios with the guys, we started the actual rollout. It was also made in a phased approach here. First of all, it was a pilot cloud. It's now used as an internal sandbox. We went all the way down from hardware preparation to the actual rollout, internal QVA. Then it was handed out to Merentus support and our gaming team started using it, providing us with some valuable feedback. Based on this feedback, some of the number of changes were made and the scope was finalized. As it was done in a more or more way, all the customizations were made in the field plugins. The remaining clouds rollout didn't take much time, as you can see here. It was roughly 10 days per cloud for the whole lifecycle, starting from a bare hardware, up to a handover, all documented and quality assured. So that was the build phase while building it. That results here. As you can see, there was four clouds deployed. One is now used for internal sandbox. The one was initially rolled out as a pilot one. One is used for staging and two of the clouds are used as productions. All of them were initially deployed as MVPs, as you can see the side is now that large. Now the workloads are being migrated and these clouds are being scaled out. We'll cover it a bit later. From the architectural perspective, it looks quite similar to the reference architecture with a set of plugins on top of it. The open-wiz feature and provider networks were used for Neutron. And what's really pleasing for me while the clouds are being deployed, Wargaming Team by itself prepared their own Morana application to integrate with their configuration management database. There is a set of policies, how hosts should be named, what kind of IPs should they use, so these policies were enforced by Morana application and then after the instance had been deployed, they are registered in the database as well. I noted to get it up and running on time, we also added training model to the actual deployment. It was a custom training based of our existing offerings. One is focused on vanilla open stack, covering its architectural overview, what's under the hood, how does actual provisioning workflow look like, how to travel, shoot it, what to look for, what config lights, help guys to change some things afterwards. And we also added a model for meridius open stack addons, mostly for fuel and Morana here. Now discovering all the components which were deployed during these deployment. As I've put some diagrams from the actual training slides here, as you can see, they are quite deeply technical, so I personally have no doubt that the guys are now perfectly ready to run their own environment. All these our administrators are knowing and know very good now. Yeah, we are still working with the team, as now we are into the operations phase, into the sustained phase. And based on the actual tickets we have and the actual iterations we have right now, the comments is really pretty well enabled and they know what are they doing with open stack and how to make some business sense out of it. While talking about sustained, there are also three main components here, the actual distribution, the actual meridius open stack is being covered by support subscription, both for packages, for architecture, for some networking team plans and small customizations here. Plugins are also covered by the maintenance, so using the same workflow, we also help to maintain these plugins while running on maintenance upgrade on the main system. And we are also considering an upgrade later this year. So that does basically how the project looked like from the technological perspective and but what are the actual results? Thank you. Our goals being reached as a result. I think yes, there are four different clouds in different locations and the cloud platform is ready now. Our administrators are ready to work with open stack and they already working with it. For me, it's really great. I saw your trainings. It's very difficult material. And at the moment, we are transitioning non-gaming services, non-gaming workloads to the open stack. At all production test environment, all production test applications, such as a portal, forum, payment service, and other services, but now just for the one, our game project which named World of Warplanes. Yes, already working in open stack cloud. It's a great tool. That's awesome. Thank you. And we have started to collect the data for the capacity management. It's really great game. Yeah, it was actually one of the reasons to give some transparency between the Wargaming and GCore how our resources actually used by which team. So this data is already being gathered right now. Okay, which lessons learned we have and we started? Lesson first. Lesson first is pretty obvious. It was told many times during these days. I've heard in a number of presentations so far. It's just, the one shouldn't treat open stack as a b-center. It's not a, it isn't a direct replacement. It's more of paradigm shift. So you can just replace the technology with or while staying remaining all the processes and all the cultural interaction between the teams here. Lesson two. Well, we don't have any feature party so far. And if you are onboarding your existing enterprise workloads, your workloads which needs some nurturing and which are to be preserved and scaled up, there are still some show comes to be addressed and I've seen some blueprints addressing them in the upcoming release. Such as, for instance, live migration, massive live migration KVM, which might still go some issues if you do it in a large scale or one of the things which is not obvious for people with VMware background is you can just scale the instance up without any giant time. As for Mopar stack perspective, scaling up an instance isn't just a use case of cold migration. You stop it and then scale it up. It definitely needs some time to adopt. And the second case? The second lesson, be prepared to revise your business processes. If you have a plan for start the same project, you should be ready to revise your business processes, of course. And if you have the documented processes with all information about people and technologies, it will be very helpful for the start, of course. Sorry, because it will be the foundation of your requirements. Lesson third? Sure, lesson third. It was good news for us. We had one more chance to understand the plugable cloud like it's actually the right way to deploy OpenStack. As we have four different clouds to be rolled out, it will be really tough to implement the same things in quite similar but not equal environments. In the same time, each time from the stage. So some amount of time and money spent for making it a modular way in the field plug-in way actually saved us a lot of time while rolling it out. And I expect even more time to be saved while maintaining and operating them in terms of scaling out, in terms of upgrades which we are considering later this year. When you have these customizations localized in a modular way, it eases up your life quite a lot. The last lesson? The last lesson, plan operational readiness beforehand. It's very important to involve the maximum of business users in this project. The Mirantis team helped us with this case and our development engineers were involved in this project, in the development process. Yeah, they even written down their own mobile applications actually. Yes. Really great. They created connectors to our corporate systems. And yeah. Okay. Sorry. That's fine, let me do it from here, yeah. Uh-huh. And this whole project was just a first step before the Great Way. Now we are considering the several nearest steps. Firstly, we will continue to transfer our bare metal workloads to the OpenStack Clouds. It will help us to free up more hardware which will be added to our cloud, to scaling them out. We are also planning to upgrade our clouds to Mitaka this year. In addition to that, we have started to work more closely with our internal product teams to plan extra features needed by the cloud and users. Based on their feedback, we are going toward more networking services like load balancing as a service, yes, and DNS as a service too. And test sound pass feature is required our big data and development teams like DBA as a services and- Data. Yeah. So basically, that was the description of the story. I want to thank Xenia. Thank you, Ivan. And thank you for your attention. We are more, you are more than welcome to ask any questions on this case and we'll be happy to answer from a content perspective. We are done. Any questions so far? Did I understand you correctly? You said that you currently have your forums and website running on OpenStack, but the only game you have running on it is World of Warplanes. Is that right? Yes, it is. Okay. But it's the start. It's like the pilot of our project. And of course, we are planning to transfer to other applications as well. And services from other games to the OpenStack, of course. But if you're playing World of Warplanes, you could be sure that it really works on OpenStack. So you're basically already using OpenStack in this case. Did you guys have any performance issues or any other networking issues migrating everything to OpenStack or the servers were performing fine after the migration? Well, some of the performance issues we stumbled into connected to that NFS backend for storage. It definitely needs some tuning. There is some work in the background to tune it well, to tune it in a better way. From CPU or memory perspective, everything's running fine. And as for like a general networking, we've seen no issues so far. There are some with storage, but we keep working on that. Okay, another question. Did you guys migrate any virtual machines from VMware or are you basically recreated everything in the OpenStack Cloud? Yes, in the test environment. We have some staging virtual machines in the VMware, of course, and we maximizing maybe all our staging, all our stagings. And of course, within U-Part we use OpenStack and some of them were migrated from VMware. So basically from a general perspective, both of these cases are being done right now. The majority of workloads, well, the large part of it is just being recreated as for new requests. Some of the ongoing VMs, which still need to be up and running, but we don't have strict uptime requirements such as the staging loads, are being converted and uploaded to KVM from VMware, which... Thank you. Thank you too. More questions, guys. Lucie. We're finished. Thank you very much. Okay, thank you then. Thank you for your attention. Thank you.