 So hello, thank you for attending our talk and to end OpenShift for cluster deployment automation in Amadeus IT context So my name is Maria Alejandra Manueli. I'm an OpenShift Red Hat consultant working in the Ironman Amadeus project With my colleagues Vincent Bronikowski and Tiwakastan. I will let them introduce themselves Hello, so I'm Vincent Bronikowski and OpenShift Red Hat consultant and I work with Maria and Tiwakastan on this project Hello, my name is Tiwakastan. I am a senior SRI working at Amadeus. I'm specialized in cloud platform migration and operation And I've been working with OpenShift and Kubernetes since 2015. I've been working closely with the Red Hat team all along that project So before digging into the project itself, let's start by introducing Amadeus in a few words for those who don't know what Amadeus is doing Amadeus is a technology company dedicated to the global travel industry. We are present in almost 200 countries with a worldwide team of more than 19,000 people Our solutions help improve the business performance of travel agencies, corporations, airlines, airports, hotels and more So why is this project? Let's go back a few years ago to understand where Amadeus comes from with its cloud journey So we started our cloud journey in 2014, even before the very first version of OpenShift 3 At the same moment, we also started our partnership with Red Hat, working closely with them on that topic In the next few years, we at Amadeus gain quite a solid maturity in the use of OpenShift, deploying and operating dozens of clusters, mostly in our private cloud at first Those private deployments are part of an internal project called Ironman Ironman because its purpose is to provide a kind of a super yes and pass on-premise and for the long run Those deployments are currently running on two Red Hat products OpenShift 3 is currently on 3.11 and OpenStack Then in parallel, we also started to extend our capacity in the public cloud, both on Google and Amazon Then since mid 2019, we started deploying OpenShift 4 in the public cloud, mostly in Azure And we're quite impressed to see how the installation, the upgrade and the overall management of the platform has been transformed, simplifying our OpenShift model for both D1 but also for D2 As you all know, the major event of 2019 is the COVID-19 crisis And Amadeus being in the travel industry business, we have been impacted So we could not grow our Ironman private cloud as in Tundin, which became quite a blocker for Amadeus's migration to the cloud As there was still the need to further cloudify our operation model and applications So in 2020, the low use of our classic infrastructure brought the opportunity of potential repurposed of several hundreds of servers to create a new yes cloud And due to the usage of older hardware, this stack has been called Ironman Lite And the purpose of this Ironman Lite project was to leverage existing UNO's hardware and provide new cloud platforms with minimal costs and an excellent operational model Thanks to OpenShift 4 and this to continue Amadeus migration to the cloud So let's talk about Amadeus technical requirements for this project So in Amadeus, we know of quite some experience deploying OpenShift 4 in the public cloud, mainly on Azure But deploying it on-premise was a completely different challenge And that's why we requested the expertise of Red Hat to help us on this task as the core of this project was run to Red Hat product OpenShift 4, so it was 4.6 of the time of the project and OpenStack 16 So we had some precise requirements First, we wanted a deployment model and operation of our private cloud as close as possible from the public one on Azure And this was a single way for SRE teams to manage our clusters Second point, for our deployment on-premise, we have no direct internet access So we needed a fully disconnected installation mode Where all the artifacts are fetched from internal repositories We also wanted to use an IPI installation, so IPI stands for Installer Provision Infrastructure And this to have the full cluster infrastructure provisioning self-managed by the OpenShift operators And thanks to machine and machine set OpenShift resources This was freeing us from the burden of managing ourselves the infrastructure And enabling easily great features like cluster autoscaling Then, first point, we wanted to use Calico as a SDN As we used it a lot to enforce proper network security In Amadeus, we wanted to leverage a Calico feature like global network set To represent external side block or also global network policies This to enforce some rules at a global cluster level And those features do not exist with other CNAs Finally, the idea of this project was to be able to create a full cluster with like a single command Consuming a single config file input And this reusing some automation already built internally by Amadeus To deploy some cluster on Azure So we don't have to struggle to recreate clusters and have a kind of cluster as a service model Okay, I will now let Vincent explain you more in detail about the project And the automation that has been built based on those requirements from Amadeus Thanks Thippo So the main thing we want to share with you in this talk is the experience of intercollaboration With a client and the Red Hat in a consultive project So we started this project with a week of Navigate Workshop The Red Hat Navigate is a tried and tested framework that helps our customers identify obstacles And align their business goals to deliver successful solutions So the Navigate Framework was delivered through a series of workshops Which covered a set of considerations for OpenShift And each day we had different workshops that were talking about different subjects So the project was delivered by the Red Hat as the principal leader with a collaboration of Amadeus And as a team, a Red Hat company worked as an autonomous team within Amadeus And the delivery was, has worked very well given the maturity of Amadeus in the OpenShift adoption journey So I will talk a little bit to you about the implementation and delivery of the platform that Amadeus required So based on the prerequisite that Thippo told us So we deployed on Primaris cluster which were on 3-8sit So to prevent crashes if some server went down It was the same thing for all the storage which were using Cinder in different 8sits So everything was well dispatched on the infrastructure And as a prerequisite everything is being deployed with the IPI fashion So you don't have to provision machines manually when scaling up the clusters I will now talk a little bit about the big challenges that we had during the implementation of this project And the first one was the Calico integration So it was a big challenge because basically the use case of having Calico as an SDN on OpenShift on OpenStack As basically not documentation So we had to do a lot of work and research to understand how this component could be working on this platform With specific integration We had to do a lot of back and forth testing in order to understand how Calico worked And how to make it viable with the needs of Amadeus And in addition to that after having Calico well configured We had to configure and understand the Tiger operator which was the component that basically deployed it automatically when creating clusters So in short terms the thing that we had to do a lot was diving into the code of this component To understand what was happening to make it work and make Amadeus happy about the Calico SDN The next thing that we had as a challenge is having an OpenStack on premised versus cloud provider solution So since Amadeus wanted to have an excellent operational model It was kind of complex to start from simple deployment using on premised OpenStack And iterate through everything and all the prerequisites that OpenStack can have So starting from the storage, from network configuration etc And how to deploy, for example, multi-added cluster So this has kind of complicated to have an on premised environment that was working as well as a cloud provider And for example even some really specific feature weren't working as intended And we had to do exchange with the engineering teams to have everything fixed for the next releases of OpenShift and OpenStack The last thing, the challenge that we have seen as a big challenge was being reactive with all the Amadeus assets and needs So the integration were quite complicated and with everything of that we had them to integrate with all the tools that Amadeus already possess So since they are really mature, for example, they have a tool that is called the Amadeus wrapper That helps them to integrate manifest when creating clusters So all the work we had done previously in cluster we had to then integrate everything in the Amadeus manager with all the tools And since Amadeus is really mature when they had some questions or some needs Because this project was quite new, we had to come fast with a tailored solution that were quick and elegant So they can be happy to make everything work properly So I will let Mara talk a little bit now about the automation tool and the technical speaking and how it has been implemented on the Amadeus side Thanks Thank you very much Vincent So while performing navigated workshops we do this exercise where we ask the client to identify business priorities We get them to vote on their top three and then we go deep into the explanation of why these were chosen So for Amadeus the main business priority identified was operational excellence and then efficiency So at the heart of this conversation there was a topic of automation We can see on the screen an extract of this exercise where automation was brought up several times You could definitely see that this was at the organization's mind Every talk of processes involved the question can this be automated This wasn't something new for Amadeus either They had already advanced quite a lot on the automation topic as Vincent and Tiva mentioned there was the existence of what they called a wrapper Which leveraged the OpenShift installer to add manifest like for example the Calico manifest or the two operations during the install As well as an automation tool based on Terraform to optimize the installation of OpenShift clusters in Asia So the easy adoption of Amadeus, the easy adoption of automation by Amadeus made it so we could work side by side to create a process that allowed us to spawn up clusters on one click So automation became very quickly one of the top priorities of this project So the OpenShift installation was automated from start to end So how did we do this? Let's take a little deeper on the technical side So the automation tool was based on two technologies heat orchestration and Terraform So first let's talk about E-Templates So as mentioned a yes was deployed with OpenStack and then OpenShift was installed on top of OpenStack to create a private cloud solution with infrastructure as a service and platform as a service capabilities So E-Templates described the OpenStack infrastructure for a cloud application in a text file such as the one you see on this slide This text file is leveraged to create a stack of infrastructure resources such as networks, subnet, the bastion server and others The high integration of E-Templates as an orchestration technology for OpenStack made it an obvious choice as a technology to use This allowed us to deploy all the OpenStack prerequisites to install OpenShift on top of OpenStack So now that we know the bastion server was automatically deployed by E-Templates you may be wondering how did we did to actually configure the bastion server So for this we used Cloudyneed So Cloudyneed is an industry standard that identifies the cloud is running on doing boot, reads any provided metadata and configures the system accordingly So this allowed us to configure the bastion server during boot time automatically So the moment the bastion server is spun up repositories are configured as T-Bone mentioned it's disconnected installed so we needed to configure the internal repositories of Amadeus Then packages are installed, the clients and the OpenShift installer, clients such as the OC client, OpenShift client and Swift client are installed Then the installation objects such as configuration files are downloaded in this case from a Swift container And finally with the run command resource Cloudyneed launches the cluster installation So where does Terraform come in? So Terraform is the glue that brings all of this together So to give it a little context Terraform is an open source infrastructure as code tool that includes an OpenStack provider which allows us to create OpenStack resources such as for example leveraging E-Templates to create an orchestration stack This Terraform also includes an Azure provider which is why it was used as a tool to create the automation for Azure clusters So how is Terraform the glue? Well we have three main things that we needed from this automation tool So we needed to deploy the E-Templates in a sequence So for example we need to deploy first the template with the project to install OpenShift and then everything else that's deployed on the project Then we needed to create installation objects and installation configuration files from different templates and we needed the ability to variableize these files and these objects So all of this was allowed by Terraform So at the end the automation looks something like the diagram you see on the screen So if we go a little deeper on the diagram the Terraform creates an orchestration stack which is the one you see right now as a stack tenant on the diagram that deploys the project where OpenShift will be installed sets up the user role assignments, configures the code test for this project Then Terraform creates the floating IPs for the ingress and for the API as well as the Swift container that will store the installation objects The installation objects are created from a Terraform template and the floating IPs, cluster parameters, project names are added accordingly And then finally an orchestration stack which is the one you see as stack server is created which deploys the OpenStack prerequisites, the bastion server and finally launches the OpenShift cluster So as you can see here this stack deploys the networks, subnet, router, the floating IP for the server, the ports and as well the cloud config resources that will configure the bastion server So this automation tool allowed Amadeus to deploy clusters in a repeatable and repeatable way and on demand serving as cluster as a service So this already made this project really really interesting but in my opinion there's something even less tangible than technology that made this project interesting And it's the fact that this part of the project was looking to standardize processes within Amadeus So this required to create a synergy between Red Hat and Amadeus So while creating this tool there was always the conversation where would this tool be compatible with what is already done or will be compatible with the existing infrastructure and workflows So this goes along really well with the Red Hat principle of putting the client first and adapting what you do to your client's need So now we'll go on to specify the benefits and wins of this project Okay so the first point and I think it was maybe the main goal of this project is that we are now able in Amadeus to provision a production ready cluster using OpenShift 4 in our private cloud And this allows us to move traffic from our legacy infrastructure and support thousands of transactions per second on our new on-premise cloud platform The second point and it is also a consequence of the first item This project enables Amadeus to continue its cloud migration So first from the legacy infrastructure but also from the existing OpenShift 3 platforms And this leveraging all the great features of OpenShift 4 for the on-premise deployment So one of the main benefits of this project as well is the automation of the OpenShift cluster which allows to create clusters in a repeatable way using one single configuration file and on-demand Last but not least the maintenance of a good relationship with a client which is Amadeus which is a faithful and long-time client and it's cool especially during these challenging times Thank you everybody for attending our talk and I want to give a special thank you to one of our colleagues, Tiba DiMai, that worked with us along this project and the project wouldn't have been possible without him Now we'll be answering questions on the chat