 Hello and welcome to the open-stack Manila project update We're starting a little late, but we'll try to move right along as best we can If you have a urgent question, feel free to raise your hand and stop us as we go along We'll try to do a little Q&A at the end. I'm Tom Baron. I'm Ptl for the Manila project For the last couple releases And I'm Victor Martinez-Decruz Contributor for Manila perfect for last couple of two years. I guess. Yeah So first let's just quickly say what Manila is because some people may not be familiar with the problem space and background Are and then we'll talk about what we did in Rocky and then what we're going to be doing in Stein and beyond Manila is you know probably is the control plane to provision and manage filed Shared file systems across storage systems. So it's an abstraction layer Just the way that say sender is an abstraction layer above block storage and has different block storage back ends We do that for file shares It's forked off from sender, but it actually solves a new class of problems I'll just make a note or you know all the open stack projects have Animals is logos and this is the Zerilla which sounds like Manila and I think that's the only connection if we didn't do Animals I probably would have done something tried to get somebody to do something created with file folders because that's where the name Came from was for like Manila file folders. So shares So like a bunch of other open stack projects it offers a self-service Rest API when there's a client there that people use or the use horizon or whatever, but the interact with the rest API that's consistent across And simple to interact with there are about 30 back ends multiple NAS protocols and FS Cep FS native cluster FS HD FS ifs, etc in the rocky release we had about almost 60 reviewers and 50 contributors and 27 companies represented. This is the eighth release of Manila The latest user survey showed 184 deployments to just people responding. We don't know how many people didn't respond, but There's a bunch of activity outside the Manila Project proper integrating it with other other things In the rocky release Manila has an API is micro versioned so we maintain compatibility backwards always So we had I'm just showing that we had a few API changes here. We'll talk about the features that went on At the time So if we change a feature and we need to extend the API or do something we micro version it And when you interact you can you can negotiate which which version you're working with One of the features in rocky where some Enhancements to the notion of share type if you're familiar with the idea of a sender volume type it's like that or kind of like a It's basically like a template for the kind of storage you want I want SSD storage or I want a particular back-end storage or whatever it use extra specs to do it So there's a default share type that's always needed. So if you create a share without Mentioning a share type you get the default And we didn't used to show people what was the default you had to go look at a file and the file was only available To administrators and so on so it's just a usability enhancement and we can also filter on the basis of these extra specs which are the Keyword value properties that are in the share types, right? So you say This has to be SSD or whatever So we can filter by those now we Have access rules and Manila that determine who can access shares So there was an enhancement that the Huawei folks pushed They have experience in the public cloud where people wanted to annotate the access rules So we now have that ability to do that that we didn't before we have metadata on a lot of the other objects that We didn't have them there before The inspr as 13,000 folks added a back end so we talked about 30 different drivers or so we got another one in this release The We're other you know all the drivers basically end up saying hey we got some new bell and whistle or something We need to add so net apps have Snapshots show up as a dot snapshot type directory and sometimes you want them visible Sometimes don't people don't want them visible so they can control that now with the configuration option that can also use a share type that says Create it with visible or not this type of thing they also We have the idea of security services and With sifts and that app you can Set the OU now when you couldn't before With the CFFS back ends Prior that there's a volume path in the file system That Used to just be set and hard-coded and now we can configure it and that means that we can run Multiple CFFS back ends so you can have CFFS back in one and two and three and so on using the same file system behind Infinite odd supports multiple export locations per share now expert locations are Basically what you would use to mount it's like in the case of an NFS. So it would be an IP address and the path that you would mount it Just to pick but depending on the protocol. It could be different things QNAP supports the new the new version of the QA QNAP OS There were community goals for every project Not every project always meets them, but they're asked to we met ours and rocky so There's stuff like Mutable configuration some of the configuration can be marked as mutable for instance The configuration it says whether you're logging with debug or just info or something like that So we marked that as mutable now we can basically do the equivalent of sending a SIG up and getting that configuration to take Used to be you would have to restart the service which is potentially It's not going to disrupt anything on the data path that's in use But it would disrupt the service in the sense that I can't create a new share or do Extend a share or do the kind of control plane operations that Manila Offers, you know during the reload so So step in terms of non disruptive operations Testing was using a library That nobody wants to support anymore and so that's cleaned up and gone We're doing one time. Oh, not too bad. So in the Stein release We bumped the number of the release We have new Community goals So one of them is to run all gate run run gate jobs under Python 3 by default. So is probably most of you know Upstream pythons dropping Support for Python 2 I think in the next year or two releases Distributions and so on are moving to Python 3 We need to get open stack ready for that. So our upstream gate jobs many of them were we had we had already back in I think metaka converted all our unit tests and Lenter tests pap 8 tests and so on so they were running Ready for Python 3 and the code base is pretty good, but the functional tests and tempest tests and so on Yeah That was not compatible by the three and we got the fix recently. So That's another thing less that we had to cover But right now it's only tempest that we need to migrate and that will be the end for our migration, but yes In Victoria's leading that project among other things among others So if you have questions about that, she's the person to talk to Another community goal are upgrade checkers And the idea there is that all our projects will run manila status upgrade Check or whatever and Basically Say there are some config options that have changed and you need to change, you know They're deprecated now or whatever you would warn you of that before you attempt to do the upgrade We have we by the way always to keep config options around one release even though they're deprecated so you don't You know get out on the limb that can cut yourself off. You just get a message saying this is deprecated now Please change it and so on but that I'm just giving you an example It's a it's in a general infrastructure that you could put in various plug-ins For checkers that would be appropriate to your project The framework thing is committed now in our project. We have to think of Between now and the end of the release about what checks we actually want to put in there, but we've technically met the goal at this point We have some carryover work from the rocky cycle for Jason schema validation of rest queries We were just doing ad hoc validation in code This allows one to write a JSON schema for each research resource they will be the schemas will Be broken out against the micro versions because and then it will do that kind of validation This is an interesting example because we we had a Great spec written for this and we great initial work and the contributors just disappeared Probably some kind of reorg at their company or something like that this happens, but this is Work that we'd like to continue and nobody signed up for it right now So if anyone's interested in it that there's really a clear design great spec Not rocket science, but but valuable work if anybody really wanted to get involved in Manila and just we would We would love to have you and and help you along with the process. This would be a great great thing to work on Another carryover is access rule prioritization Manila has right now an ambiguity with access rules in that You can have overlapping Access rules and it's ambiguous which one will win To me, it's intuitive that the more specific rules should win other people say no, I want to be able to override the more specific rule with Another one that comes first that should take priority over it So that I can unmask it later and there's use cases around that So this is something people argued about and different back ends did it in different ways So we decided to allow users to put a priority on these And they can decide for themselves This work There was good progress on it and Rocky is still continuing. It has an owner But I expect it will get done this cycle We have something called driver handle share servers that confuses people but their driver handle share servers Has to be either true or false some of the but some of the drivers Present themselves Basically a virtual version of themselves to each open stack tenant or a open stack town is a keystone project or you know and They can in those cases those drivers spin up a new Virtual server as it were for each tenant. Those are driver handle stock share servers equal true Servers they're more complicated. They're nice in terms of The open stack multi-tenancy model But there's our features that We're present with the simpler drivers the DHS as equal false drivers that weren't there for them So we have specs open right now For work to address those gaps the manage unmanage feature is Basically a way to import shares Into Manila and open stack management. So say you had some shares you already out there that were being used privately on a Back end but for some purpose and you say hey, we want to make those available to Open stack people and you're gonna bring them under open stack management. That's called managing the share Similarly, if you want to relinquish control, that's called unmanaging the share So there's an open spec for that that we expect to merge in the next two or three weeks Barring some fatal objections. Similarly, some of the back ends can do replication of shares So and They can do it when they run in the DHS as equal false mode, but not in true mode so there's work to enable that and basically What it comes down to is the specs say we need to be able to Run We have this abstract notion of a share network In Manila, which is for practical purposes is often a description of the neutron network and subnetwork on which The VM that's going to mount the shares lives, okay I'll back up to on that in a moment, but That that's a classic case at least and that's what the share network would mean Well, if you have replication you have a segmented network You need to be able to represent the fact that you've got you've got one replica over here one replica over there And tie them together So that's what this is about was a multiple multiple subnet so so on and the In the notion of a share network that we don't have not currently Then we have three open specs that we expect to merge For work that probably would get done in the cycle or at least get started in the cycle and by the way, I I Don't get all hung up on doesn't get completed in this cycle We start stuff and we we work on it and if it doesn't then we carry it over to the next one That's I know we're very waterfall six month cadence But some things need to be done like that and other stuff just needs to get done As soon as it's ready so There's the idea that you can create a share from a snapshot and Currently that's limited To doing it on the same storage pool as the original Now what if you have other storage pools or back ends that are actually Consistent with that snapshot so that you could conceptually Created on the others then you'd like to be able to make use of that storage There's a spec about that it's tied up right now into arguments about what the policy ought to be for that Do you assume? Local cow when it's fast and want to do those first and then optimize for speed and efficiency of space Or do you want to spread it and how would you represent policy for operator decisions and so on? I expect that'll get worked out in the next couple weeks, but we'll see it's an interesting one There's improvements in the idea of availability zones Availability zones as you know are kind of just labels for Places you put stuff, but generally the intuitive ideas that would represent some kind of failure domain so there's specs to Improve the way that when you schedule Shares or place shares you take availability zones into account and these thoughts and There's a spec that to expose different kinds of capabilities that were currently not exposing to end users so that they can select the right kind of shares So that's the work that we had specs for But the overall theme From my perspective as PTL is the these are all great But really where we want to Improve Manila is to make it more easily consumable and more easily adopted and that means Not just by end users doing point and click on a rise them and so on although That's a very important area and there are things you can do with the CLI right now that you can't do with Horizons, so that's one gap to make it more usable so make it more usable for humans We're doing it, but also to make it more usable by programs By platform as a service and so on now one thing I'll say about Manila one way that it's very different than sender Sender serves storage up Through a hypervisor. That's essential to it And you have to go out of your way to do tricky things to for instance make it work for bare metal with ironic We saw a lot of talk and some of the Keynotes and so on about developing infrastructure that's not just for Nova right well Manila was never really just for Nova It serves storage up over a network It doesn't care if the consumer is a Nova VM Bear metal Container workload running on a VM container workload running on bare metal Manila doesn't know the difference and That makes us very versatile and useful for a world of Open infrastructure that can be with Nova or without Nova So We have some gaps. However, we're not we're not in the open-stack client currently We're not in the open-stack SDK currently you have to use a Manila blah blah blah instead of open-stack and We're not in that SDK. We're trying to address both of those things in this cycle Open-stack Ansible has started working to incorporate build some roles for us in there The Kubernetes right now is successfully using Manila with the dynamic external service provider You'll see it. There's a talk from CERN where they're using that external service provider with a CSI driver on the node For Cephapas is back in we're looking at Working with them and other people to extend that so that we have a CSI on the master as well Master as well as on the nodes to use Kubernetes terms for it and or master and worker mixing my CSI and Kubernetes but and These are areas that actually I think are going to be as important or more than adding new bells and whistles within Manila itself Because Manila is pretty mature works our Work is in the Stein cycles being tracked on this wiki anybody can look at it. We have forum later this week Where we want operators to talk to us about what they need and help us calibrate whether what we're working on actually fits with real needs But that shouldn't be limited to one little conversation at summit This is out there. We encourage people to reach out to us and talk to us whack us on the head with two by fours Say why are you working on that? Nobody cares about that. Here's what we really need You know, there's so many things to build we might as well build something that's useful You can find us You know, you can use the open stack developers and operators mailing lists, which are merging by the way, but solved update that slide and You can find us some free node Email me email Victoria We'll get you in touch If you're interested in contributing to Manila there are many ways and there's an onboarding session here the Wednesday and That's kind of it. I think we're running short on time. We did start late ourselves Because the there's a cumulative effect here, but let anybody have any burning questions or Also be available outside Maybe we can step outside and let the next group start up try to get back on schedule here and