 Welcome to our presentation, The Sandwich Situation. We would like to do some introduction into Ansible modules and how to combine them with the OpenStack SDK. My name is Niels Magnus and I have the great pleasure of having my colleague, Artem Goncharov with me, who is also a cloud architect in the OpenTelecom cloud and also deeply involved in several OpenStack projects including the OpenStack SDK. Let's have a look on what we are talking about today. Most of you are probably familiar with Ansible in one way or the other. So configuring software with this framework is pretty easy. For provisioning of cloud resources like servers, volumes, networks and stuff like that, often other tools come into your mind like OpenStack Heat or Terraform or whatever else. But as we want to highlight here in this presentation, it is also quite easily possible to configure and create and manage resources with Ansible itself, especially with so-called Ansible collections. We have a short example here how this could be done and we'll show a more in-depth example in a minute. The overall architecture is that we write a playbook with invocation of Ansible modules, internally called the OpenStack SDK, and the SDK itself translates everything into API calls, gets the results back, parses these results back to Ansible. So that is the big overview. Can we go one more slide? Here we have described a little bit the Ansible flow of control and maybe that's a good point to explain how this works. Could you do that, Atom? Yeah, sure. So, how Neils already mentioned, playbook in Ansible is a set of individual tasks that actually describe the state on the target system which Ansible should try to ensure it is really currently. So the playbook is a list of individual tasks. For each of these steps in the playbook for each individual task, Ansible is copying from the controller host to the target system where it should be executed or the stage should be ensured, the module itself into some temporary directory. What might be interesting here to say is really that example. If we would like on the remote system to ensure that the user exists, for example, then we clearly need to connect to this remote system. While when we are trying to ensure something is present on the local host, we are doing this on the local host. So anyway, in the concept of Ansible, there is a connection between the controller machine from where the playbook is being orchestrated and the target machine where the action is actually executed. So going in our workflow further, there are different connection types possible in Ansible for the connection really to the target machine. If this is a really remote machine, Ansible can connect to it using SSH, even FTP is possible and all the other weird and unweard possibilities are there. If the target system is actually still a local host, if we would like to ensure something on the controller, then we might be using a different SSH or whatever the other possible. So basically what Ansible is doing is copying the module to the target machine into temporary directory. On the target machine, it invokes it using a separate Python interpreter, passing it all the variables which are described in the playbook. In our case of Ansible, or not Ansible, sorry, of OpenStack provisioning, the module in our case, the OpenStack module will invoke OpenStack SDK, which in turn is actually triggering all the final API calls. What's going further? Basically, the module is passing the result of the API invocation after it retrieved the result back to the controller or to the Ansible playbook utility so that it is capable in using those results in some further steps. Maybe we should do a brief example on that. As you can see here, this is the installation and the invocation of a very simple playbook that we explain in a minute. So in this example, we just go to our home directory, create a virtual environment, activate it and install both Ansible itself and the OpenStack SDK. So this works for every Linux machine, for example. If we want to issue commands that actually work on the OpenStack SDK itself, we need the Ansible collection for that and this is called OpenStack.cloud and can be installed with Ansible Galaxy. Then obviously we need to create a playbook explaining what kind of resources we want to create or modify or manage and then we can invoke it with the last line here, Ansible playbook-i inventory, which is just listing the systems that we want to manage or in this case, localhost. I think this is an optional parameter and in our example we coded this directly anyway. Then we need to pass the playbook and the interpreter itself. And it's very important to understand that this interpreter needs to be present on the target machine. So but since in our example here, the target machine is the localhost, so that is just fine because we just created this virtual environment and in the virtual environment there's the binary to the Python and everything that we need there are prerequisites, especially the OpenStack SDK is installed there. So that is how it is invoked and now let's have a look into the playbook itself. We want to provision a server here. We provide a name using OpenStack Cloud Collection. We are going to connect to localhost because that's the system from where we orchestrate a future cloud server and then we have a number of tasks. There are one, two, three on the left-hand side and two extra tasks on the right-hand side and a special one at the very end. So we start with provisioning a network and pass just an identifier cloud and a name for that to the network. Then do the same thing with a subnet and with a router and as you know those are prerequisites that need to be prerequisite resources that need to be available before you can provision a server itself. So once we created those free resources, MyNet, MySubnet and MyRouter, we can provision the server itself and use for that the module OpenStack.cloud.server and provide all the attributes for that. So and that's it. And as an example of doing the other way around for retrieving some information from the cloud, from cloud resources we have the two remaining invocations of module and please take a look at the name of the module here. It's OpenStack.cloud.server underscore info. So that factors the server details of the virtual machine VM and passes it to the internal variable server info and displays it with a special command debug here. So that's the whole magic, how to use the SDK within a playbook. And now let's see how this is implemented internally. So actually in Ansible in the collection of OpenStack and actually this is pretty much everywhere in Ansible there are two types of modules. One is info module which is basically there just to fetch information about the remote resource or about the really the state on the system and the action module which is creating the resource or updating or deleting the resource. Let's have a look quickly into the one example of info module taken out from our collection, from our OpenStack collection. The module is for basically fetching information about volumes about cinder volumes. I will not be going through the details really very deeply of the how Ansible modules work. Therefore, if you would like to contribute or if you would like to get really more deeper understanding of how the modules are structured please follow some of the links that we will post. We will have later in the presentation. But just very, very briefly Ansible requires that your code your module is bringing also valid documentation with it. So that's what we see at the very first line documentation. We are not describing the content of this documentation actually because it's really long and long and long and Ansible really requires that and forces this documentation really to be valid one. Documentation is the documentation of the code first or code is a documentation first however you would like to name it. It also there is a possibility to describe return value what is expected the module to return what is the structure of the response and potentially also an optionally basically an example of the invocation different ways how you would like to how you can invoke the module. What we do then the next is pretty much familiar to everyone who is having not only Python but any developer experience is basically we are importing a class definition when importing OpenStack module itself. This is more or less our entry point how the modules should look like and what we do further we declare we define how the basically what what our module actually is. And the first what we are actually doing is we are describing in the code which parameters this module is actually supporting. So in our example the volume listing volumes or finding the volume from the sender point of view is supporting the following parameters details whether we would like for example to get information about the volumes with deeper details or without details whether we would like to list volumes for all projects in the domain or just for current project project whether we would like to list only volumes with this name for example find find particular volume would be clearly specifying its name or whether we would like to list all volumes which are currently available in available state or vice versa all volumes in the error state. So basically we describe to the Ansible which parameters this module is supporting what is the type of each individual parameter whether it is required or not whether it has default value which in values it is supporting and so on and so on. After we have described to Ansible which parameters are supported let's go through the function itself that does really the action. And you see this is a very cool example of how easy can it be with the help of OpenStack SDK really to get information from the cloud. We have a more or less not more or less but literally a two-liner. In the first line in the run function we are invoking block storage volumes function from the self.con which is more or less establishing connection to the cloud that we have chosen and this is coming already pre-connected by inheriting from OpenStack module class. We are passing into the call all the parameters that we have that we potentially have converting it explicitly into list since the SDK is returning when you are listing resources. SDK is returning Python generators and therefore it's more or less kind of like required to convert it to list before we are able to return it by Ansible to the invoker. And the next line is actually what we are doing is we are telling okay Ansible I am the module and I am ready I am exiting passing you the following JSON information. Volumes is equal to results and which might be least might be single entry whatever it is. And at the very end we have some kind of pretty familiar to anyone bootstrapping of the module. So nothing really weird here. So it's maybe a good good place to point out that if you intend to implement something like this yourself you are to stand on the shoulders of giants and actually leverage the code of the SDK it makes sense to familiarize yourself a little bit with the SDK itself this is why we have provided the link to the documentation here in the example. But how to implement now an action module. And this is exactly it. So we see pretty much familiar stuff from the previous slide we have documentation, return, example we are also inheriting from I have forgotten to have an import but we are all pretty smart people we will be able to find it out. We are inheriting from OpenStack module which are the supported arguments by this particular module and we are currently looking to the real example of volume backup module. The module which is creating or deleting or managing really volume backups. Pretty much new stuff compared to the previous slide we are having additional stuff that's Ansible or this module is supporting. There are some different weird ways of using Ansible in case this parameter is set then I would like that some additional parameter becomes also mandatory or vice versa or whatsoever. So in this particular case required if we see a pretty much confusing construct but it can be really very simply described as a following if state is equal to present volume then the list of parameters including a single value volume is required that's it there is nothing more and the next what we are doing also is we are describing to Ansible whether our module is supporting check mode and check mode in Ansible is more or less something like a dry run so the module is expected to evaluate whether it should do something but it is expected not to do anything in reality. So just to evaluate whether the backup should be created or not but not created. So what we are looking on further next function so far we are not looking into from where the function is involved we are simply describing the function which is responsible for creation of the backup. The very first step what we are doing here is we are evaluating if we are in check mode more or less return saying yeah we are going to do something what we are doing further we are extracting some of the parameters for easier access in the following codes change is equal to false is more or less starting the value default value and what we are doing further is we are trying to find volume in order to create a backup you need to have a volume that you are going to create a backup of so volume is really a prerequisite here. We then also specify some initial attributes that we will be passing to the backup like name is equal to name volume ID, force is incremental and so on and so on all the attributes that are supported by the open stack on the on the cinder side by the backup next page is a basically a continuum of the function responsible for creation of the backup if user has passed a snapshot to us we are searching for the snapshot in the cloud exactly also to be able to pass in to the create backup snapshot ID we are injecting metadata description whether they are set or not and the most important step here more or less in the middle of the slide is the invocation really of the creation of the backup creation you see backup equals self con block storage create backup passing all the parameters though pretty much we are done here but what if the user invoking the backup will become available since pretty much every following steps every following step might require that the the backup is not simply there but is really available for example if you have a volume you can't create a backup of the volume until this volume becomes available and the same with snapshot so what if we need to wait until the backup becomes available the user can pass in the parameters wait variable or wait parameter which is actually by default just for you to know is set to true and in this case we are more or less waiting until the backup would become available you see pretty interesting construct wait for status passing in backup status available basically describing exactly the expected state and we are telling that we would we are allowed to wait for the maximum time out number of seconds so if we have waited and we have achieved the backup that the backup becomes went into the status available we are exiting with change to call true telling to the answer bill or to the invoker side that yeah we have done something and this is your result volume backup is the structure ID as an additional data otherwise if we got an exception that we are simply raising raising exception further in this particular case not really raising an exception further but in signaling to Ansible that we have failed with the following JSON data and then if we were not expected to wait we are still having our regular exit JSON which is more or less also returning the same data now the next function with create we are done what if user would like to delete the backup this is the function that is responsible for doing that and pretty much in the same way if we are in the check mode return true if we have the backup then let's delete the backup here as you see in the code and again we have pretty much same construct if user would like to wait until the backup would really disappear not simply that we send an API call to delete it but we really need to ensure that the backup has disappeared so pretty much everything same and at the very end exit JSON changed true pretty much done and then the main function the main function which is invoked by by the Python interpreter our entry point into the module name equals so again extracting display name just for easier access while it might be not even necessary but in this case we are doing that what we are doing since we are taking care about backups and user might have passed us the name let's try to find this backup and then we have a basically a choice if the stage if the requested stage by the user in the playbook was present so meaning it is expected that the backup is present if we don't have a backup but we are expected that the backup is present then let's create a backup so just involve the function we have seen previously otherwise it may basically important when we are willing to might be update the resource what if for example the name of the backup is changed or the description of the metadata or whatsoever so this is normally the place where update functionality is taking place however in this current in the current situation SDK doesn't support updating of the backup and basically someone even say this is not really necessary I would just rather delete the backup or recreate it and that's it so that particular in this particular example we are not updating it SDK doesn't support so what we are doing is basically we are immediately exiting telling to Ansible we have not changed anything and then the next couple of lines what if the expected state is absent well just delete the backup we have it just deleted and that's it then more or less that's we are done the very same bootstrapping call that we have seen in the previous example and yeah we are ready this was easy yeah that was the implementation of the module and now we just compiled you some extra information on how to place this Python code and how to package it all together so one good starting point is the documentation link on the right-hand side first one which explains how the OpenStack Ansible collection is implemented and gives some extra information about that on the directory structure you can see how and where to place the files that we just described and if you want to learn how to well create you have started already and to package to test and to install those collections that's what the second link is for and if you want to dive deeper into module development and find out the requirements of Ansible itself in return values and the passing variable interface that's what the third link is about and that's it basically and we are done but before we are concluding the presentation let me briefly wrap up everything so Ansible is powerful as a configuration management system and it can also be used now with Ansible collections and a special case of the OpenStack collection as well as provision infrastructure as well modules are the blue part between Ansible and the SDK world and it's packaged as a collection as an Ansible collection it's very easy to install them into an Ansible environment and invoke them directly modules can also be implemented easily thanks to the existing OpenStack SDK so take a look at this project if you're interested in extending the Ansible collections with the resources that you frequently use and that's why we very happily welcome new contributors to this OpenDef project and the OpenStack community at large and we hope to welcome you there soon and to start contributing to one or the other project thank you very much before we close the presentation and open up for questions let me briefly point you to our OpenStack scavenger hunt so for the occasion of the ten years of OpenStack we at Dutch Telecom started a small scavenger hunt where you have to solve ten small not too hard puzzles and if you find out the right places where to find the secret passphrase you can actually win some gadget photo drone if you're interested just there's the link and we'd be happy if you participate on that thank you very much if you have any question ask them either right now or send us an email and we say thank you and ask the question thank you guys