 Welcome to my workshop OpenStack Client 101, becoming a command line power user. Today I show you what are your options to access your OpenStack Cloud, how to install and to configure the tools for that. So my name is Nitz Magnus. I'm an architect for the OpenTelecom Cloud at his systems. And I also do some community outreach management. That's why I'm talking to you today about a selection of tools that we have for that. So let's start with how to interact with your cloud. So first thing that comes into mind if it gets to accessing a cloud is usually a web-based console, a browser-based console that runs in your web browser. And this most of the time looks like this. So you may have seen this on the right-hand side. This is the classic OpenStack Horizon dashboard. On the left-hand side, we see a slight variant from that, what we have here in the OpenTelecom Cloud, but basically you get the idea. There are several resources like servers, like volumes, like networks, and you create and manage those resources with your mouse in the browser. It is a very convenient and familiar meta for the actual resources. They are kind of a one-to-one mapping. But there is one fundamental issue with this approach. It is not very easy to automate. So if you have ever tried to script the mouse path or clicking on a specific item, especially on complex UIs, that is not very convenient. That's why we have a number of different options. And I'd like to discuss some of those options with you. So the first and the most basic approach would be to call directly the API. So we have on the right-hand side, the cloud with an API, which is exposed to the internet and can be accessed through HTTP or HTTPS. To do that, we could facilitate a tool like Curl, and I've put here an example, a call to that and also the response. With that, we send a token as a header, that's dash H and access just a URL with some endpoint URL and a version and an API core. And that results in some data, which is usually in our case in OpenStack environments in JSON format. This is an array of servers with their tenant ID, their addresses, and probably lots of additional attributes as well. This approach can be used all the time and then under any circumstances, but as you can see with the variables, there are a number of pieces of information that needs to be filled in. So first of all the token, which is some seemingly random string that verifies your identity, which you get after your authentication. And the second thing is here the endpoint, which needs to be determined because a classic OpenStack cloud comprises not just of a single API endpoint, but several of them because each service has their own and they need to be figured, can have different versions and so on and so on. So there are lots of questions to deal with like I just mentioned here. So should you use a direct API connectivity for your automation? Well, obviously you could write a script that facilitates the curl commands and that accesses the raw APIs, but keep in mind that you have to do a lot of housekeeping like the authentication, like the service discovery and so on. So this is not the really realistic option unless you want to spend quite some time in implementing that. More convenience is provided by an SDK. The OpenStack SDK is an official OpenStack project and maintained by the community. And it works in between the developer and the cloud itself. It directly talks to the APIs but takes care of dealing with the data and providing some programming language interface. In the case of the SDK, that's Python and I provided here a short example script. We import the library. We connect to a predefined cloud in a configuration file. I get back to in a minute and then we can just access the list of servers, iterate over it and print it for example. It's a very simple example. Well, this is a very powerful and generic approach and it takes care already on several aspects like authentication, connection handling, the marshaling of data and de-packing and so on. However, you need some, let's call it intermediate Python skills for that. So to get that running, well, there's a lot of documentation available online but some background is certainly helpful. Some remark regarding the SDK, the SDK can be extended. Extended if you have extra features in your cloud that surpass the classic OpenStack API. I just briefly mentioned it because we have a couple of extra services like extra database services, for example, RDS is a database service. The way how you access those extensions is just the same as with the classic SDK. You connect to it and then you have different objects for that resource objects that can be accessed. More details you can find in the documentation. Nice thing, the plugin mechanism checks automatically with any further do if there are extensions installed and everything else is taken care of by the SDK itself. Okay, so then let's have a look at the current situation that we have in the OpenStack ecosystem. So as you all know, there are several modules available, Nova for compute, Glance for image handling and several others as well. And all these modules provide APIs and each API obviously needs a counterpart. And that's why several projects like the Nova project provide directly a tool that has the logic how to access it over the API has built in. So you get one binary more or less directly maintained from the Nova project and there's everything inside. Other projects like the Glance project have factored out already the functionality from their tool and they now use the functionality of image handling which is part of the SDK. That is a separation of concerns so that the team of Glance just needs to provide information about the API handling that can be also done by the SDK itself. And the Glance client more or less just calls the appropriate parts in the library. That is a more scalable approach. We have some problems and issues with this approach because the different tools might have some slightly differing semantics and features. Some have output formatting, some don't have these features. The UI is not really consistent and each tool is implemented individually by the different teams of the what you would and services. And that's why a couple of years ago the OpenStack Client project or also called Command Line Interface was started. With that we have one single command OpenStack and then as a first parameter, the service or to be more precise, the resource and then some kind of verb what to do. This is typically one of the CRUD operations. So you could invoke it as OpenStack server list or OpenStack image list or OpenStack network list and so on. It's not only the list command that is available but usually you also have also verbs like show or delete. And sometimes depending on the resources obviously also modification options. There's an extensive help function built into the OpenStack command. So please consult that. So you could also run something like OpenStack server dash dash help or OpenStack dash dash help and you will find specific information about that. The CLI tool, the OpenStack Client is automatically aware of potential extensions and those extensions it also extends the command itself. So to stick to the example that I mentioned earlier you could do something like OpenStack RDS cluster list or something like that. The output looks like I've presented here in the gray box. And you can also notice that there's already some formatting done here in those ASCII art boxes or tables. There are a couple of options for that though. Yeah, so to summarize the idea of this tool so we have one tool for all services. It's very convenient that it takes care of all the authentication and data transformation issues which are shared by all services and to achieve that it relies on the SDK. To install this project it's pretty forward if you're a little bit familiar needs only a little bit know how about Python and what I recommend here is Python virtual environment so you don't change anything on your controller system. You need some Python free, you need free packages which I recommend that are available on your system GCC, SSC compiler, the libraries and header files and the virtual environment which is not installed by default and on all Linux distributions. Obviously this works directly out of the box on a recent Ubuntu 2004 LTS but should be available on most other distributions as well as slight modifications. Then I create a virtual environment named P3. I activate it and then you just have to install in essence OpenStack Client as the basic package from the PyPy repository. In my case, I also installed the VL and OTC extensions packages because then everything gets compiled and it's a little bit faster. But those five steps done, you can issue OpenStack-help and use that. Please keep in mind if you lock out and lock again, lock in again after some time or another day to reactivate your virtual environment. So that means to re-issue the command in step number four before you do any work with OpenStack because otherwise it won't be in your path. If you want, you can do this directly in your startup scripts, bash RC or whatever you use. Yeah, that is half of the story installation. And next is authentication. There are lots and lots of configuration options for the SDK and the OpenStack Client. And they more or less fall into three categories, Command Line Switches, Environment Variables and Configuration Files from which I recommend the letter, the configuration file. So it's very easier to put everything in a single file and not have it on the Command Line because as you know, you can see your co-users Command Line options with a PS command. You can even see environment variables on the same system. So especially if it comes to passwords and secrets, it's not a good idea to have them in Command Line Switches or Environment Variables, but to put it in configuration files. The configuration files have those long list of potential keywords and attributes, as you can show here, but the minimal requirements are listed here in the lower left-hand side. But the cloud you give a name to an instance, username and password are quite obvious. The domain is necessary, a project might be necessary and what is necessary is a first authentication URL where the client connects to figure out the actual service endpoints for the specific services. So typically this is your Keystone URL. There are some extra parameters as well. I won't go into much detail here, look up at the documentation. There are a couple of documentations and examples. One important thing, there's obviously a precedence. So if you have something in your configuration files, you can still override it with the Command Line or the Environment Variables. Here I have a full example for a Cloud Siaml file. On the left-hand side, there is the actual file and you see I have created two instances. The first instance is called OTC and the second is called MyProject. And those are two different accesses to my cloud or could be even two different clouds. Where I provide username, project name, the domain name and I also provide here a profile. A profile is the preset of exactly those configuration options which is stored in a file called vendor JSON. And there are some commonly shared attributes are passed there already like the authentication URL which is always the same for all instances of one cloud provider. Another thing that you can personally factor out are the credentials themselves. So you can have a second file optionally that has the same structure and has the secrets in it and that should be in the same directory as the Cloud Siaml file. You have three options of the Clouds Siaml file and the secure Siaml file. The first is the local directory that's obvious. The second is in your home directory in dot config slash open stack. And for global use, you can also use ETC open stack. So the local directory has obviously the highest precedence. Now as we have talked a lot about installation and configuration, let's talk briefly about output. You can select parts of the data. You can choose the format and you can also use some extra commands to pretty print your output if that is necessary, especially if you get a lot of data back like a list of all servers or network resources or something like that, then such a tool might come in handy. Let me give you an example how to select columns. As you've seen in the previous example, there have been four columns and if I just want to have two of them, I can put in a selector dash C and select what I want here. I can also change the output format. So if I use the show command, I usually get way more attributes and it might be more convenient to have them in JSON format, as you can see here. And especially if you enable the JSON format, that then it makes sense to pipe it through JQ dash capitalize C dot, which means take the output, do some nice formatting. It's already indented here, but it also does some syntax highlighting and the dot means select everything from the very top. You can think of JQ being some kind of a grab command for structured data. Have a look at the documentation for that. Yeah, let's wrap up before I go into some live demo mode if you want to summarize everything, the command line is a powerful tool that helps you to automate your processes managing your cloud. I really recommend to stick to the standards and not throw everything on your own. Stick to the SDK and the OpenStack client, don't reinvent the wheel again. We've seen this several times, that's why I'm stressing this. The installation is relatively easy. Once you get the hang of it, it's just a combination of Python, virtual environment and a single PIP invocation. And for the configuration, keep into mind to write a proper CloudSyaml file and you can select the different instances with either a command line switch or an environment variable so that you can easily switch from one CloudSet up to another. You can customize your output. And with all of that, I hope you have learned now about a powerful tool. And hope you have much fun and successful. Thank you very much for actually close, very brief hint on some competition we currently run at OpenTelecom Cloud. There is the OpenStack scavenger hunt open until the end of the month. And you have to answer 10 not-so-difficult questions about 10 years of OpenStack. Visit some web pages, find some secret characters and with all that, you can win some cool gadgets. I think it's a photo drone. And well, let's see if you want to participate on that. That's it. Thank you very much for your attention and maybe there are questions or remarks for that. Yup, bye-bye.