 Okay, I guess it's time Good morning, everyone. My name is Yahya Bay. I Work for HP work HP cloud private cloud. I work closely with the HP public cloud as well, and I work with the Glossomar Aguiar and right here just to help us out here In we're basically trying to solve the domain quota issue As you probably all know You're gonna find figure out. What is what is this about? Today with Keystone 3 We introduced, you know in grizzly domain domain concept was introduced There's a lot of Work that has to be done to to get domains into Fully functional and fully featured one of the things that we were having An issue is is the lack of controlling the The quota of each project today basically all the resources the resource consumption for cloud admin is is basically Done down at the project level and You've got People that create projects a lot of projects and Use resources that don't really have control over where the cloud admin needs to control that at the at the source level so What are we trying to do here? We basically wanted to add some kind of a sport at least in the In Havana to to have to give control and quota At at the domain level and so we can we're going to continue to support Quota right at the at the project level at the service level, but it's very important that we We try to solve this problem for all of us So what we try to do is we wanted to keep The quota Management at least at the project level and for example Cinder Nova Quantum or networking they all have quota driver to To to manage resources, but they're really not It's like a almost an afterthought there is no control In any time you only you need to manage the Changes or or capacity for any project you kind of have to go to a tenant level or project level you don't have that control at all, so what we're trying to do here is add Basically the quota set to basically manage the limit right at the driver level in At the project level, but extend it over to the domain in The other thing that we wanted to keep track of the usage of this quota, so you have some kind of control or managed in Domain usage set limits are basically if you go in and use capacity your capacity and you're not tracking the usage It's going to be very difficult to To find where resources are being spent So that's what we're trying to do basically add The extend the quota over to the Domain level added to Keystone or or some other mechanism, but we have some ideas though I'll gloss on we want to add anything. No good Okay, so we're not changing the current quote quota sets. You've got These quota sets will essentially be the same we're Again the what we wanted to do is add the quota driver in In each service so for example if If you have quotas already set in for Cinder Cinder capacity volume number of volumes or And do you want it to that's at the project level and a domain sets quota right at the domain level what we're going to do is Validate that the quota with the domain level first and that will take precedence over the quota at the project level Meaning that it's a configurable option It's up to the cloud admin how they want to use the resources and if they want to give control to the project Itself to manage these quoted. They can still do that There are various aspects to this we talk about some rules and and Control that will happen at the domain level. However, we want to be able to maintain Addibility with the existing driver so we were we're not forcing the domain quota at To everyone at everyone, but it has to be sort of a configurable option We we need to make sure there is a correlation between The domain quota Enforcement with the project level the drivers and how do we do that? There are several options Do you have anything? Yeah You have done so, so basically what we are trying to do is Adding the possibility for setting quota at the domain level so as of right now as of today You can set quotas at the project level, but there is no way to to set a limit at the full domain level So we have for example, we have the the the folk water that whenever project is created that gets Assign it directly to the to the project and set the limit for those project, but If you create as many project as you want you can easily consume the Entire capability of your private cloud So if a domain represents for example a department and the cloud administrator wants to limit the capacity of each Department or organization he cannot do that as of right now because he cannot set quotas at the domain level So basically we are not trying to remove The existence of the project water, but instead we are trying to we are We are planning and suggesting to put another level of control that would enable the administrator to actually set limits in a higher level and in the next slide we Understand that there are some constraints and some relation between the The project usage So the domain the domain there all the consumption happens inside a project So setting a limit at the domain would mean that that limit would Set the limit for the consumption above all projects inside that domain Although there is still a limit at the project level. So I guess that's what this This is life is trying to To clarify Okay, so So there are several options one of the option is we Basically Add Apis to Keystone In the engine within Keystone that manages just the quota domain quota and on top of that we layer basically a Driver for each project. There is a there is a nova driver today for quota for example We're not gonna throw that away. We're just gonna Have a lightweight driver that That basically does all the requests Reservation rollback all all the operations that you would do with the current driver with With the with the quota Manager driver so basically if we wanted to keep the original driver and you don't have domain quota enforced In configured you could everything works as as normal today But as usual, but if you wanted to enforce the Domain quota management you can enable that in in Nova config or Maybe we'll have a we haven't really done all the details on how to implement the the configuration but it's a configurable option regardless and so the option is that the domain quota driver from each project with Send requests to The domain quota measure within Keystone There's gonna be a Keystone a couple of things we're gonna change in Keystone one is We have to add We have to add the quota Schema in and there's gonna be a couple of tables one is the domain quota table and the other one is the usage table so we keep track of where things are and basically the rest usage here is Basically the domain driver would call a set of rest API's there are within the Keystone But the other thing that's not sort of The the typical crud operation that you would do as an admin from let's say From the client API whether it's horizon or something else. So horizon you'd have crowd operation you could create modify Modify set quota, etc from your horizon, but These operations are basically The Once once the operations are set the the control goes into the domain quota management engine or service within Keystone and Everything is handled basically as another layer of the current driver any anything you want to add or That's one option, right? Right. So so that's one option. So basically what I was explaining is We need a crud operation for the quota in Keystone set quota for a given domain Get quota usage and that kind of stuff and we also need a rest usage in which these these extended for a driver needs will communicate or connect to in order to Get reservation commit consumption grow back consumption and that kind of stuff So this is and this domain quota driver is basically an extension for the existing domain Sorry for a day for the existing quota driver that already exists in the service So the idea is before proceeding to the project Check-in what a check-in we first make sure that domain as a whole you still have Enable have it have the capacity to allow the It's a quota check you have a question It's fine. Okay the I wanted to keep this picture for a second and this is Rest API rest usage is is a little there are catch to it right performance. How do we handle? Request from multiple Services so you have two or three four novice and number of novice and number of Cinder and you only have a single Keystone. So so that that sort of puts in a little Pressure on the resources another option, which is Sort of flows with the current open-stack model, you know use the a MQP so message it messaging and The It's still in Keystone Rest API are are pretty much the same credit operation. We're not changing that That has to be done. So the admin can control The extension and but there is a Q Qn option and I like this approach, but You want to talk a little bit about that? Sure. So so the idea is that I mean we still need to The capability to have synchronous calls in order to make sure that I have do I have enough capacity? Should I proceed? Yep, you do so the idea there is is to build the synchronous message in which you can you actually inform a call back to the message and then whenever the Request or messages possess it it can give you back Response Right, there's a catch to this one The catch with this one is that currently in Keystone there is You can't really do messaging. It's a security. There is a blueprint We can reference that right then That has to basically support secure messages and We can't really implement this this approach with the current design unless the blueprint is approved so the alternate option which is Create a quota service. It's a Basically Domain quota service or manager is a standalone Keystone will have the As we register the service with Keystone all the current operation can still be done through The crowd options will be in the quota manager not in Keystone So it's it's really a nice approach where we can Still manage the the quota for domains in Keystone with the with the leveraging of the AMQ option in Glossamer and there must be also some some integration there whenever you create a domain and The limits for the domain should be Automatically set in the quota service. So there is some relation between Keystone and the quota service in that case as well Okay Before I move on any questions Will wait okay the rest is basically it's It's really little details on how we can accomplish this We can talk about The kind of issues that we have to deal with regardless synchronization the quota check reservation. How do we do that? We have to deal with with roll Rollback and if you have failures, there's a lot of contentions that that having the domain service by itself sort of kind of not have to deal with the Keystone specific issues Because For example in order to come up with these options for implementation We had some some ideas in mind for example The first one how could we be as less disruptive as we can so then it came up with the idea of the driver The domain quota drive in that kind of stuff So so we wanted to to make sure we understand the idea and the need because we actually believe that this is absolutely required whenever you Want to take advantage of the domain feature in Enterprise level so Yeah Again when we started doing this we were gonna extend the existing project service level drivers in With that we've we ran into all kinds of issues It's it's not a It's not a simple problem to solve, but it's not impossible to solve either so in currently Having each project deal with with its own driver the same way they want it to do for you know Oh, we need to do was quota. It's an afterthought Glance doesn't have quota and I'm not sure it belongs in glance. There is all kinds of Project level or the service level quota that shouldn't really belong there It's really you want to manage your capacity and set quote quote a cloud admin need to control that at a higher level and that's why it's having a separate service or quota driver for a lightweight coder driver that fits into every service or like a plug-in makes Makes the quota In at least feature in it's extended to the future With any new open-stack service or project or resources. It's not the end. We have to continue to do that in and Really control the quota management In as a separate as a separate entity, so that's what I'm thinking Okay Yeah, what what we we can go to questions But what I wanted to show is that we have an implementation. These are the the quota setting operations you've got the post to get the put and delete boilerplate type type of credit operation And it's the second in here's an example, right? You probably seen this before and if you look at the quota today The service at the project level It's basically gives you the tenant ID or the project ID and the quota that is available And I'll give you an example. Let's sender gives you sets quota at the tenant level project level for The capacity and the number of volumes, but if you have three three projects you you kind of see that now You could have multiple Project, but you can see it right at the at the domain level and So we're controlling that again at the domain level. It's very similar to what what's done with the project But it's actually at the domain level sequence diagram we thought of that's Not as Interesting What do you know you want to talk about this or no just questions, okay? Just showing the current quotas with Swift Network and so we can talk about Open for questions. Yeah We get over there. There's one in the minute. Yeah So Thank you very much. This is great and something that's very needed and I think you're on the right track This is awesome a few things. Why why why are we sitting in this room for this session? This is a design session. We where's Dolf, you know, that's Totally agreed. The problem is you try to sort of submit sessions and they'll approve it and say look It doesn't belong here doesn't belong there and we need to know sort of it There's a consensus about this do we and then sort of if we have the buy-in from The contributors and from the public then they say, okay, we'll bring it into design and we'll sort of that's that's the approach That's why I came in as a design discussion and it turned out in a journal section. Oh, okay. Well, I've got your back So And who are you? I'm sorry. I'm sorry. I'm Everett tapes from Rackspace. Oh, I've got the centralized quotas Design session this afternoon at 320. Oh, excellent. Yeah, please. Yes. Can I can I count on you to be there? Bring this deck Let's maybe condense it to about, you know, five to ten minutes Sure, and you know really focus on the the options I think is where we want to focus and let's get some some action items out of it and right Let's get a blueprint out of it. We have a blueprint Already it it's just I need we basically the whole idea is come here discuss the quota. Oops. Sorry, and In the blueprint print is over here already, so we we already have it we have a Sorry Anyway, I have a blueprint. I have an ether pad So we've got all that discussion the whole idea coming here is what to do the design discussion I've done the discussions for other things, but But it it's fine. I take I take anything I can get so we can come into that and push it through Yep, sounds good. I'd love to do that. Great. Yeah. I'm Excited to share the session with you, you know, just be right up there with me. That'll be great. Okay Couple other things. I think you guys should think about Including project level quotas in there. I know that they're already in the individual projects. Let's Start moving away from that So when you're when you're doing your design You know, I like I saw the calls in there and Stephanie You know, there's all domain and you know, you need to develop what you the features you need So, you know start with the domain stuff But, you know, haven't I keep keep in mind that, you know, you might want to also do this at the project level as well Yes. Yeah. Yes Okay, good Or at least don't design yourself into a corner Yeah, I mean, you're not gonna really understand how it works until you've got some sort of proof of concept So I mean start with the domain stuff, but you know really keep in mind that oh, you know The very next thing everybody is gonna scream for his projects and this is this is great work That's that's very much needed. The last thing I want to say is you might also want to Did you think about quoted defaults at all how that would work in this? Yeah There are a couple of things one is that let's say we we coexist with the exist with the project quota and That's one of the things is that we don't want to be disruptive that's the approach that we could take and If you said domain quota Obviously, you're gonna honor the let's say you have project level quota that is higher than the domain quota Then we basically have to Make sure that we only honor the domain quota, but talking about default quota We are gonna have a set of default quota and we were thinking what could that be right and so that's that's in the thinking process right now in but Yeah, not necessarily just to know that you guys are thinking about it. Yeah, absolutely. It's being given consideration Yeah, okay. Thanks a lot Yeah, so Any other question? Yeah One is So the other question You do have But you also The usage that's the usage enforcement, so we are Monitoring that right anytime you sort of Use the quota that's get it into the second day You have multiple drivers So so the idea here was to be able to all the the attributes or Capacity or everyone on the call that you already meet at this service and project level You could do that at the main level so number of instance number of CPUs The idea is to support the same set that is already supported today for novice in the Networking didn't take a very deep look at swift You had Yeah So, so you have the limits and you need to control the usage, right? So in order to make sure that the usage does not Six, I mean does not go higher than the limit. So that's the Exactly, but but if you I mean right and no so so we have the keystone, right? So that in that in that design. We have the domain quota management at keystone I don't want at every quest for for reservation or something to go down to novel Can you give me all the resource for this domain? Can you give me all the resource for that domain? That's that's the point behind the driver, so you wouldn't really need that I agree, but again, we can Have mood for Nova so the domain can spend all of them So what Nova does not know about all the usage of that domain because there are other other Nova services in the the system so we need to catch there in a single location. So there is yeah, you have a question So Yeah, did you think about for one of your options instead of constantly going back and doing a read from Keystone or separate put a management service To actually have the quota returned in the validate token call, which is already happening. So it'd be available The token validation occurs right as in the middle where it's in front of service It gets rolls back. It could get token back as well. You could do that. And then it's one less What happened to the usage That you can't really control the user Thinking about the metadata right we could receive the the limit, but from from my users perspective How would we do we still need to to be able to reserve and then commit the consumption? But you get in it from Nova or from Cinder you're not getting it from You could do that. Yeah, it's an option. That's a great option. We'll just Any other I think we're we're good. This was a design type of topic that I wanted but obviously It's we just have to put it out there and I'm glad you guys are Thinking about it in HP cloud one Hopefully behind this But how do we we go from our so? What time is it again? 320 we'll be there B1 14 Great. Thank you for your I think we're done here. Thanks. Thanks everyone. Yeah, you're welcome