 Sing some song. Okay. Hello? Hello? A song? Okay. Okay. Cool. To use that one. Yeah. Thank you. I wish you more than I have to. Well, good morning. I see it's a very hard morning for many people, so. Okay. When you will want to evaluate decisions, you can visit this link. And now, please, welcome new presentations about Open YPA. Can everyone hear me? Okay. Good. So my name is Audie Lee, and looks like there was a pretty busy night last night, so let's look around. But I've been working, I work at Red Hat, and I've been working in the dog tag certificate server team for the last few years. And I've also been working as a core developer for the Barbican open source project. Barbican is a secret as a service, and that directly ties into sort of the identity work that we're doing here. I'm Rob Crittenden. I've been working on Free IPA for, I don't know, six, seven years. I dabble in OpenStack, and so that's about it. I work at Red Hat also. So a lot of the work that we're going to present here is done by the members of the team that we're involved in. And so the other members of these team have been pretty active in OpenStack as well as in Free IPA for a long time. Adam and Jamie are both Keystone developers, and have worked a lot on a lot of the different parts of what we will present here, which was part of a demo that we did for the Tokyo Summit. Rich, who is here, and will bail us out when we have questions about this, worked on the original IPA join service, which Rob is going to talk about. John Dennis worked a lot on the easy to SAML integration stuff. Robbie Harwood worked on MariaDB and Curburizing MariaDB. And Nathan, who's our manager, is a Keystone developer as well, too, and will also be someone that will bail us out if we can't answer anything. So it's pretty straightforward agenda for today. We're going to do a brief introduction to IDM or IPA, a very brief introduction to OpenStack. There have been a lot of presentations over the last couple of days for both OpenStack and IPA. So most of the concepts and everything should be pretty familiar to most people here. We'll then talk about how IPA fits into OpenStack and the types of integrations that we're trying to do. And then finally, we'll talk a little bit about the proof of concept that we did in Tokyo and hopefully demo at least some of what was there. So, Rob. So who here is familiar with IDM, IPA, so I can kind of gauge how, okay, so I'll go over the details. All right, so it's basically identity management system. So you're dealing with users and user groups and hosts and groups of hosts and managing access control for all of those things and storing them centrally. So at the heart of IPA is an LDAP server, we use 3D and IDS, and we have a number of custom plugins that do things like synchronized passwords between the LDAP password and the Kerberos password. So you have a single password everywhere. We use MIT Kerberos for our KDC. The PKI is dog tag, the product that Audi works on. We have bind integrations, so LDAP again is the back end for bind. The CLI and UI have the same framework, so anything you do in the command line, you can do in the UI and vice versa. So let me talk about the architecture a little bit. So I'll go back. I'm just testing you now. Let me talk a little bit more about this. So client here, okay, so we don't do anything fancy. We're just gluing a lot of things together. If you've ever installed Kerberos yourself, it's like an all-day affair. You probably get it wrong four or five times. And so the goal of IPA was to make this consumable, so it's one command you install and everything is ready for you, and then you just have to concentrate on users and groups and things you care about, not, you know, where do I get my next principle from? So the client here, which isn't shown really, is SSSD, is system services, security demons, something like that, and it does most of the grunt work that users see. So it has, it's caching like NSD, and it has offline support, and it's honestly what most people think of when they think of IPA. So it also enforces host-based access control, so you can say this user or a group of users can do certain PAM things on this group of hosts or host, and that's pretty powerful because then you can start saying that administrators only can log into this set of hosts, and maybe they can only log in every other stage so they can't do other services. There's also a lot of web integration so that you can tie into the PAM stack and have PAM enforce only certain users can access this remote service over the web. But again, this is all client stuff, and I'm a plumber, I did all the server stuff. Let me show you an example architecture, can you go to the next? So one piece that I didn't talk about before is active directory integration. So in the past, we did syncing where we'd sync the AD users over to IPA, and you could use them that way, and there was a way to share passwords between AD and IPA. That's problematic because active directory administrators tend not like to install things on their domain controllers, and they don't really like other people poking at their database. So instead, we're using AD trust. So to active directory IPA looks like just another forest, and it's all pretty seamless. Now right now it's only one-way trust that works, so active directory users can log into Unix systems and do pretty much anything any other user could do. We're working on the other way, it's a slow, hard process. But typically this is the way it looks. So an administrator may be an active directory administrator, he may not be, he can use a browser in access, it all sort of seamlessly. This is what, here's the client. So we have SSSE as a client, CERTmonger is a really cool tool that's used to manage certificates. Okay, so dog tag will issue certificates for IPA users, and CERTmonger can do that request for you in one command, and it can also do renewal. So you never have your certs expiring it to in the morning on Christmas, which they always seem to do. So if you get nothing out of this talk, CERTmonger is something to look at. So and then the business applications, these modules I was talking about, these are the, Jan Paziora has a number of modules too. One can look up identities out of SSSD. So you log in as yourself, and then you can get all your groups in Apache, and they're set as environment variables, and that way you can enforce business logic that way without having to go to LDAC yourself and get the data. It's just all there. And then like I said, there's some PAM access modules and some other cool stuff. You can go through an IDP, IDP is a SAML concept. Does anyone know what SAML is? It's another single sign on protocol. I'm not going to get into it in detail because I don't want to use the entire talk, but this all sort of feeds together. So you have your users from AD, from IDM, and then you can just do your work, which is probably what you guys care about most. So that's IPA in a nutshell. So we'll talk a little bit about OpenStack. How many of you are familiar with the, with OpenStack and the components of OpenStack? A little bit more. There have been a lot of talks about OpenStack, so I imagine. But basically, OpenStack is cloud software, right? So the mission statement kind of says exactly what you're trying to do. It's to create a cloud computing platform that meets the needs of public and private clouds. And the idea, of course, for it being simple and scalable and all that good stuff. OpenStack is basically a set of components. Each component has a project name associated with it. So I'm just going to go through a couple of these because you're probably familiar with these. So Nova, for example, is the compute server. This is the one that you use to create servers, whether they be virtualized or bare metal. And of course, in order to do that, it needs images, which it gets from glance. Swift is an object store which is scalable and redundant and so on and so forth. Neutron provides networks. Cinder provides block devices. Heat provides templates where you can orchestrate putting in all of these things together and the deployments and so on. Cilometers monitoring and so on. Keystone, which you may or may not be as familiar with, Keystone provides authentication, sort of a central authentication store for everything. And in particular, it has sort of a directory of users and groups. And it issues tokens as a result of this. So what typically will happen is that Keystone actually is configured to use an external identity provider, something like IPA or some other mechanism. And it gets the users and groups from that identity provider. And in the Keystone database, it is actually just mappings between those users and groups and projects and roles or tenants and roles within OpenStack. And so when someone goes to Keystone to authenticate, they'll be redirected to that identity provider, they'll authenticate there. And Keystone will use those users and groups to map to tokens, which it will then pass on and then it will try then passed on to the different services to allow you access in different ways. Horizon is just a dashboard that allows you to control things, it's a way to UI. One thing that's not on there that is useful, that is the one that I work on, is Bobbican. Bobbican provides secrets as a service. So for example, in the case of volume encryption, Cinder will store volume encryption keys for volumes that you'd want to mount to different VMs and Nova will then go and collect those with the appropriate Keystone token, will be able to retrieve those encryption keys when it actually mounts the volume. We'll talk a little bit more about that as we go through. So just briefly some kind of an interaction between these, that's maybe a little harder to see. But in the center, you've got the compute node, which is Nova. And of course, Neutron's going to provide networks for Nova. Again, Glance is going to provide images and block storage from Cinder. These images and so on will actually come from Glance who might actually store those images, will actually just catalog the images and store them in Swift. And finally, you're going to add Keystone that's going to provide identity to everything over here. So each of the OpenStack services has a fairly generic type of structure. At the top over here, you've got an API calls. You have a REST interface where you can make calls to handle, to manage each of the different resources that are there. On this side over here, each of the services is also going to talk to a database, they're also going to talk to a messaging queue. And finally, at the back end, they may talk to some other service, or they may talk to some other devices. In the case of Cinder, this could be local storage, or it could be a Swift storage. Swift store, in the case of Barbican, for example, it could be IDM, or hardware module or something like that. So it's a very generic sort of overview of things. But what's going to become important because when we look to secure some of these services, we're going to be looking at securing each of these individual endpoints. So we'll want to secure the REST API on top here. We're going to secure this communication here, this communication here, and certainly anything back here. We've probably had a number, we've had a number of talks about Triple O as well too. The basic point over here is that Triple O is open stack over open stack. And the idea is that you're going to have a small bootstrap deployment network here, which is just usually typically a single node that has all the open stack services there. And you're going to use, and that's called the under cloud. And you're going to use the heat templates over here within the under cloud to actually deploy the cloud that you desire up here, which is the over cloud. So that's the idea of Triple O. And so for example, over here on the left hand side here, we've got our under cloud, our under cloud has each of the different open stack components inside there. And it's going to be using the heat templates over here to deploy different kinds of nodes inside the open, inside the over cloud. You might have a controller node, controller node will have pretty much all of the services there. Or you could have more specific types of nodes like the compute node, which would just run various NOVA instances as well as storage nodes, like a Swiss storage node or a Seth storage node or some other kind of storage node over here or a block storage node, which runs sender. And the idea would be that you could scale this as large as that your cloud becomes. So typically, for example, you're going to want to have three controller nodes for HA and those will be configured in an HA configuration to allow redundancy and so on. So that is briefly an introduction to IDM and to open stack. Let's see when they kind of join, they fit together. So there are three possible scenarios you can sort of think of in terms of how to fit IDM into open stack. The first idea is that you have IDM as a standalone component that you can use to secure the infrastructure, both the under cloud and the over cloud controllers and the nodes. And I'll be talking about that, and that's one of the things it's actively been working on right now. The second is as a standalone component, which you can use to secure access to the VMs and the servers that are created by the user in the over cloud. And Rob's going to be talking about that. And then the last one is finally, if you have IDM, the IDM server running in a VM that's created in the over cloud. And then you have other VMs that are created in the over cloud that need to join to IDA, IPA as a client. So in this first scenario, you're trying to secure the infrastructure using an IDM server. So the idea is that you have an IDM server that's standing by itself. It's in the infrastructure itself. And we're going to use it both in the over cloud and in the under cloud to make things a little more secure. The one thing about OpenStack is like everything else, everything was developed without security because people want to get stuff done. And then as a result, at the end, you have a wonderful cloud that can do all of these wonderful things, but that is completely insecure. And so IPA is one way of, has the tools and the ability to be able to make a lot of this secure. So the first thing we're going to do for each of the nodes, whether they be controller nodes or in the under cloud or where the over cloud is, we're going to register the host as an IPA client. And we'll do that by running an IPA client install on each of those. And this allows the host, it'll create a host entry inside the IPA database, which will allow the host to be placed in the host group. And when you place the host in a host group, you can assign things like pseudo policy or HBAC, host-based access control policies. And you can do all of the things that Robert mentioned about in terms of these policies. So you can basically specify who can go into these different hosts and what they can do there. It also allows you to retrieve key tabs, because it'll configure Kerberos on these different hosts to talk to the central Kerberos and get any kind of key tabs and user certificates. And we'll be allowed to be able to authorize you to do various kinds of IPA operations. And one of these kinds of operations is the retrieval of certificates. So there are a number of places here where you can use certificates in order to secure things. These are three controllers over here. These three controllers are arranged in an HA configuration. So at the top level, you've got all of the services which are actually exposed through HA proxy. And so Keystone, Cinder, Neutron, all of those are exposed by a single point there out in HA proxy for the REST API. And HA proxy uses a pacemaker and so on to rotate the different controllers over here. So if we have one controller goes down, the traffic will be redirected to the other two controllers. In order to secure that connection, we need to use TLS. As you get TLS, we use Surtmonger to go to IPA and get the certificate. So typically what you would do there is on the undercloud machine where you're running the heat templates in order to create these different controllers, you would run Surtmonger there, get the certificate, and then you would copy the same certificates and keys all three controllers. Then on each of those controllers, you can ask Surtmonger to track the certificate so that in the event that it expires, or it's getting close to expire, one of these controllers, say the master controller as it were, will go through and try and renew that certificate. And then the other two controllers will retrieve the certificate from the specific entry within the IPA. So that makes sure that you're on Christmas day, 12 o'clock on Christmas day, your certificates will expire on you. On the backside of it, the other thing we can do is we can create service users for each of the services, so Keystone will have its own service user, Neutron and so on, and we can retrieve certificates through Surtmonger for each of these different users. And then we can take these client certificates and we can talk to either the rabbit MQ, the bird queue over here, the message bus, or we can talk to the database and we can secure those connections both using TLS. And the videos, we created a couple of videos for the OpenStack Summit in Tokyo, and one of the things that Adam Young was able to show was in a default configuration, you could go through and you could see all kinds of passwords and all kinds of plain text things when you talk to the database, if this wasn't secured, because the password, when you're connected, you could see the pass code and plain text and so on, and that's never a good thing. So you can either use Cobra or you can use client certificates to secure these different connections. So just to recap a little bit, you're going to create service users for each of the different services, you'll use Surtmonger to get the Surt for the HA proxy, you're going to secure the connection to the messaging queue using TLS, using the service user certs, that way the messaging queue knows exactly who's connecting to him and they secure the connection. Cupid, which is what we actually ended up using in the demo, we used Kerberos to secure as well too with varying results, and you can also secure the connection to the database using TLS, using the service user certs. You could try using Kerberos, there will be a Kerberized MariaDB available in the next release of MariaDB and full Kerberos encryption will be available following after that. It's possible to use Kerberos as well in order to secure these. And then finally, just keep in mind, Surtmonger keeps track of all the certificates so that when they need to be renewed, they will be renewed automatically. The other thing, of course, that you can do is that because we have host-based access controls, we can specify who has the ability to access the OpenStack services in the cloud. And so, for example, this is very useful if you have a private cloud. For the under cloud, you want that to be a very restricted set of people that is allowed to get there, so maybe certain lab admins or the only ones that are allowed to get there. And you might want a larger group to be able to access the over cloud controllers. So, for example, within the corporate IT, IT itself would maintain the under cloud and then the over cloud could be made available to company-wide or to all of the developers in the company or something like that. Corporate IT, in fact, very often, let's say, that all the users and groups and so on are in AD. As Rob mentioned, AD controllers don't particularly like people going in and messing with their entries, but the identity can be federated through IPA, through trusts. And so, a lot of times when you're putting in an OpenStack deployment and you're bringing it into an environment, there are authentication stores that are already there. People aren't going to want to duplicate all of those users and groups within Keystone or within IPA necessarily, but you can federate the identity through here so that makes it easier to deploy within an existing infrastructure. That's kind of nice. The other thing that you can do is that you can set up web SSO and single sign-on and CLI single sign-on through using either Cobra or SAML. And I'll just go through a couple of running out of time. So we'll go through just one of these. So, here is a case where, I'll actually demonstrate this hopefully at the end of this, but here is a case where someone is logging into Horizon and one of the things they can do, so they can get a Cobra's ticket beforehand. That Cobra's ticket goes through their browser and they go to Horizon and they want to connect into Horizon. Well, Horizon at that point can redirect their browser to Keystone, and then you have various Apache modules that go through and process the request. So, model.js API retrieves the user's ID and so on. From the Cobra's token, we'll go through here. We use model.com identity to retrieve from the IDM server to retrieve things like the groups and the roles for that particular user through SSSD. And ultimately that information, the users and the groups get passed to Keystone where there's a mapping between the groups and the users to specific projects and roles. And there's a token then that is then issued by Keystone and sent back. So then the token over here is sent back to the browser, which gives it to Horizon and then Horizon can use that token to determine whatever it is that you're supposed to be able to do with an OpenStack. There is, here's another one, which I'm not going to go through now, but this is basically the same idea using SAML. The difference here is that you have model of Mellon, which is doing the SAML work over here and it talks to something like Epsilon, which will work with IDM to retrieve the users and groups. You still use model.com identity in order to get the group information and what you'll end up with is a SAML assertion and that SAML assertion will then go to Keystone. Keystone will determine from that SAML assertion will know things about users and groups and so on and ultimately create a Keystone token, which will be sent to Horizon. So if you want to know more about Epsilon, there's another talk later today by one of the developers for Epsilon and so I highly recommend you to see that. The third bit of the infrastructure has to do with Bobbican and as this is a project I work mostly on, I'm pretty excited about this, but Bobbican is the basic idea behind Bobbican is that there are people that are going to want to store all kinds of secrets, whether they be passwords or encryption keys or any of those kinds of things and this provides a central location for people to be able to store them securely. Most often a lot of people try to do crypto and they do crypto badly and so the idea would be to store this in a place which has been written by cryptographers, as it were, to make sure that everything is secure and so in particular we've used it for doing things like encrypting volumes, encrypting images, some of these are ongoing efforts right now but some of these have already been completed, encrypting objects from SWIFT in transit, setting up Neutron albass, load balancer as a service, so on and so forth, so various things that are various open stack services that are talking to other services and storing secrets and retrieving secrets. Where IDN comes in is that Bobbican has a back end plug-in infrastructure so that you can talk either directly to something like an HSM, a hardware security module or in our case I created a plug-in that allowed it to talk to the dog tech KRA. The KRA is just one of the components of the dog tech certificate system that is used to store secrets securely. It's also part of something called the IPA vault which is a feature with an IPA that allows you to store secrets. It's later to be in OSP9 as tech preview so coming up pretty soon. So just to give you a little bit of idea about how Bobbican works here, basically someone wants to store an encrypted volume, Cinder will actually ask through Casteland, lost Bobbican and the KRA to generate a key that key will actually stay there and what will be returned is just a reference to that key that reference to gets kept in the metadata for that particular volume and then when Nova needs to come and attach that volume it will get the metadata, retrieve the key and then attach the volume and from the point of view and it actually attaches in the hypervisor using a Luxe encryption in the hypervisor so that from the point of view of the volume it just looks like any other volume and I'm definitely running out of time. Okay so Nova has got a number of hooks available for the last couple releases and so we're utilizing three of them so when a new VM is created by Nova or when it's destroyed by Nova or when Nova detects that there's a network change so the idea of this is so that when Nova creates an image it's pre-enrolled and IPA so that when the user finally gets it that's booted and everything they can log in using their own credentials and just use it as they would any computer they might use and it can be enrolled in when the host is created the idea is that it's given access to one or more host groups so then you can control access at the point where the VM is created about who can log into it and what they're allowed to do there so what happens is when a VM is created as hook is called we create a host in IPA we created DNS entry in IPA for that host and create a one-time password and that's passed into the VM during boot using cloud in it so at the end of cloud in it packages are installed IP client is run and the host is enrolled so at that point it's ready to go for users so you can log in as yourself do everything you can you know in any other system and then when so when you're done with that because VMs come and go and opensack all the time what novel automatically delete the host from IPA clean up the DNS entries and everything is fine so the host class is mentioned here at the bottom so IPA has this this concept of auto membership so you can create rules that say member a user a class of this should automatically be members of these host groups so you're not limited to a single you don't have to you can have more complex configuration you're not limited to a single host group per per type so you can have you know very complex rules and this includes sudo so you know some VMs may have more suitable access than others and some users also may have the host only gives you so much access but users still are constrained by whatever other rules there are so this is what I sort of went over this is basically what happens with the hooks I'm not going to go over it again sorry alright so here's so there's another twist so open stack has a concept of metadata and there's like 18 different names from meditated and open stack it's parameters and you know user data and all kinds of other things the basic idea and this is going to be in mataka in a big way so that so glance has got a metadata service so you were planning on creating an IPA metadata service so that when you created an image or you create a flavor you can associate IPA with it and say hosts created with this are going to get a certain class so that when you boot it they're automatically given whatever access that you want there's also a way to override that there's a next generation launch instance which is sort of a wizard and so it's unlike what you're probably used to it actually walks you through all the steps so you're less likely to forget something and it'll actually warn you if you do things that aren't too bright like it'll see you get a little a warning if you try and boot a flavor that's not big enough for the image that you've selected for example so and it's so it's very configurable and we're hoping it's it's the way to go so rich did all the work I'm standing on the shoulders of giants here the demo to Tokyo summit so if you want those videos on it already if you wanted to see this in action not sure when this is going to land and I was in OSP at all we're but soon we hope ah oh I guess just mentioned DNS so this is the one big question mark okay so right now we rely on IDM for DNS and you know it looks like administration may not want that they may want to use designate which is the DNS as a service so we're trying to figure out how to do the creation and deletion of hosts so that that's indeed the one big open question yeah and I'm just going to briefly mention the third case here so the third case is a purely you know I might mention the old cloud use case this is a case where you have IDM or IPA as within one of the opens one within one of the overcloud VMs and then you have other overcloud VMs that are going to connect to that conceptually you think that this should pretty much work as is after all whatever is in the overcloud shouldn't have to worry about what happens in the end cloud one big question of course is things like DNS and and how that works and whether or not that there's anything that we can do to make it easier for various overcloud VMs to join IPA but that's probably something that's a little further down the line so in Tokyo we just basically had a proof of concept here there's a whole lot of details on here that I'm not going to have chance to to go through but you can kind of see all of the different things that we described and was able to demonstrate in this open in this thing we just had a single node basically with all the different servers and all of those connected to another node which had IPA and in particular there are a whole bunch of different links here specifically to videos corresponding to those different nodes I can try to do a bit of a demo I just don't know how much time so whether to do the demo or to do a just ask questions so yeah are there any are there any questions well let me yeah so we have a we have scarves and we have a keystone developer t-shirt with keystone t-shirt over here so the best question gets a really cool so yeah so the best keystone question or it's a really cool t-shirt we have swag and then we have we have we have scarves so so let's go with questions and we'll see if we have any time yeah so in integration of IPA and things I think is slated for OSP8 something that was a good keystone question did everyone hear the question and the answer did it make sense the question was what can I do with IDM today and the answer was you can't do this but you can do LDAP off so keystone can hook into a number of different identity providers you can hook right into AD you can open LDAP IDM you're just not going to get Kerberos and typically based access control any other questions good scarves no one's a scarf no one's a scarf anymore all right all right one minute we have one minute see what I can do here okay so what I've done is I've so I'm using export here and now I'm just going to log in as admin so I've set up Firefox over here it's going to take my I have a Kerberos ticket now because I've got a KNIT and I'm going to go to IPA as an admin and what I plan to do here you can see obviously that I don't get prompted for any username or password anything like that because my Kerberos credentials have been proxied through and what I'm going to do is I'm going to go ahead and I'm going to create a use a new user with an IPA so we'll say no and his name will be say for no demo I'm going to give him a password which will he'll be prompted to change on first setup okay and I've added that user yeah because it's a it's not the actual password it'll get it'll be forced to recess on the first on the first usage right so let's go ahead and I'm going to go here and now I'm going to KNIT as Brno okay I'll put in that one character password and now it's going to ask me to change the password and actually put a real password in there with all of the password requirements and so on okay okay and now you can see I have a credential over here what I'm going to do now is log out here okay and now I'm going to log in again and this time I should be logged in as the Brno user while that's going on I'm also going to log in to horizon okay so you can see here I am as the Brno user you'll also notice that I am in the IPA users group and for OpenStack I'm going to say let's log in using Coopers and so what's happening here is that horizon is being redirected to Keystone it's getting various users and groups and using that Coopers credential that's there and what you're going to see is that I'll be in horizon right here in the demo project and I have various accesses and I can at this point I can create VMs and so on and so forth the reason I can do that is because in Keystone there's a mapping from the IPA users group to the demo project of which I'm a member and I'm out of time all right Thank you for presentation so so so I don't know how to say it in Greek, but if it's fast enough, you don't have to turn it on until the end of the game. We'll have to play both. Yes, let's come up with a presentation, which is from Modbuk. You have 40 minutes, the questions are included, of course, in Czech. For a good question, you can give the bowls. You can use the three, the six. I'll show you the bowls when they arrive in time. Stickers, false speakers. And if I can just ask for the presentation on the bottle? I think we'll play the whole game, as it's a podium. And I don't know anything else. I think even if there's a person here, we'll see. I don't know how to say it, but we'll see. So how are you going to say it, but you won't see it.