 So, hello everyone. Welcome to the session on the end of the day on Thursday at OpenStack Summit Tokyo. We want to thank you for being here today at the, pretty much at the end of the summit. This session is Manila, an update from Liberty. So, my name is Akshay Parthasarathy. I'm a technical marketing engineer at NARA, focusing on all things cloud computing and OpenStack. I'm here with Sean Cohen, principal product manager at Red Hat, and Tom Bechtold, OpenStack cloud engineer from SUSE. So, this presentation is about Manila. So, here's the agenda. So, we'll first cover what Manila is. We'll then get into why you want to use Manila. And as part of that, we'll run through a couple of use cases for Manila. There are a number of different use cases, so we're going to go through the most important ones. Sean is going to touch a bit on containers and Sahara as well, since those are hot topics right now. Liberty updates, the main topic of the session will be the, will be next in the agenda. We'll then go through distributions integration, specifically with Red Hat and SUSE, using RELLOSP and SUSE Cloud. And after that, we'll go into new features that are coming up for you in Mitaka. And I will do a demo of a state-of-the-art feature called Share Replication at that time. We'll open up the questions, we'll open up the floor for questions after that. So, just to jump right into the content, what is Manila? Manila is the OpenStack file share services program. Manila is for file shares, what Cinder is for block storage. It dispenses in a self-service open-res API shares to tenants of the cloud, or those that even reside outside the cloud. With Manila, you can provision, for example, a one terabyte self-share or a one gigabyte NFS share. Do authentication with Active Directory if it were a self-share, and set up permissions appropriately to shared networks. So we're dealing with shared file systems in Manila, unlike Cinder, which is just block storage. So there's a networking component that is involved. You want to make sure that, you know, when you, there is, you want to make sure that the appropriate networks have connectivity to your shares when you provision them, and there's a bit more magic in Manila that goes, that goes on behind the scenes to make sure this can all happen seamlessly for you. So a bit of history. So Manila began as a, the talk around Manila began in May 2014, and at that time the program was overflowing with a lot of interest, and we continued working on it in the six-month cadence that is the norm for all open stack programs. In June, we submitted it for consideration as a core service, and in Kilo, we actually, in June, we introduced it as a capability, and in Kilo, we submitted it for consideration as a core service. We went through cycles of continuous integration, and we put in a significant number of new features and bug fixes in both those releases. So that's just the history, but let's just get to where we're at today. Manila, most importantly, right now is production ready. We have a total of 14 storage drivers available for you. The number of blueprints or features completed is currently at 51 for Liberty. This went down slightly from the number for Kilo, but we also fixed a number of additional bugs. The total number of bugs fixed was 184. So there were a number of major blueprints that were completed, and we'll go over all of that later on in the talk. Some of them are share instances, REST API, micro versions, experimental APIs, and so on. So Manila provides a number of different deployment options for you, and there are obvious benefits to it. It gives you a lot of flexibility to have these different deployment options. The first one is you can have Manila mapped as a single storage virtual machine or a multiple storage virtual machine, and what this basically means is that you can either have Manila manage the life cycle of your share servers or you can let Manila manage storage resources on a share server that you have already set up. This is set using the Boolean variable that you see there in the configuration file for Manila. There are also a number of different network plugins that we have. Right now we have a total of three plugins. There is the standalone network plugin, the nova network plugin, and the neutron network plugin. Each of these can be configured with a number of different segmentation options, and this also gives you that flexibility that you desire when you deploy Manila. So here are a couple of charts with the code contribution and the reviews. All three of us, Nadab, Red Hat and Susar, are major contributors to the Manila program. Nadab has been a pioneer and a leader in the Manila program since its inception and continues to be doing so today. For the Liberty release, we've had a number of new contributors to Manila, including CloudBase, Fujitsu, Scalabee, NEC, NTT, and Let V Cloud Computing. So you may be wondering, okay, all this is fine, all the details about Manila, but why do you want to start using Manila today? And the reason for that is the spending on file-based storage services is expected to reach north of 34 billion in 2016. And this, you know, the diversity of applications that we have today depend on something that can be performant, something that's scalable and provides simplicity in management. And as we are all aware in the OpenSAC Summit, OpenSAC provides that infrastructure as a service capability on which Manila can provide these file-based shared services. So the question really becomes, why not use Manila with all the infrastructure as a service capability that's already offered by OpenSAC? Here are the different use cases. We have standalone shared file services management, enterprise applications, DAVops, Sahara, containers, database as a service, automation and integration with the Manila API, HEAT, hybrid cloud shares, and many more. So we'll cover through some of these in the next few slides. So the first one is standalone shared file services management. So when we think of file services, file shares and the way they were dispensed in the past, you typically have one or two people in a back office. You know, when a request comes into IT saying that I want a one gigabyte self-share, a person in the back office in your IT department is going to go ahead and dispense that self-share. You know, he's going to run a couple of Perl scripts to do that and then, you know, make it available for the end user. Now, the problem with that is if the person decides to leave the company, then you lose all that work that they have done with, you lose the consumer interface and you also have to reinvent the entire infrastructure. So you want something that provides an open, ubiquitous REST API that Manila offers and you want the same level of self-service that was there before and Manila gives you those things. The other thing I want to point out is Manila offers this in a completely vendor agnostic framework. You can choose the storage vendor that you want and as I mentioned, there are about 14 storage backends that are supported today. The second use case we have for you is enterprise applications. So some of you may already be aware. One of the major reasons why people want to switch to OpenStack today is simply because of cost. So you may be using a virtualization technology that's proprietary in nature, not open source and it's really expensive. So if you take advantage of what OpenStack can offer for you at a cheaper price, then you have a pretty compelling value proposition to make to your decision makers. The other reason is that Manila provides that file share service capability. You don't have to move applications over to an object store or a Swift interface. You don't have to re-tool those applications. So provides that capability out of the box. DevOps is the number one use case for Manila right now. Manila provides this pluggable infrastructure and this is enabled by something called a storage service catalog within Manila. With the storage service catalog, you don't necessarily need to have your archives provisioned on SSD storage. You can choose the storage that you want for your particular use case in DevOps. So you can have your archives mapped to low performing dust if you prefer that. Analytics and containers are also really great DevOps use cases for Manila and Sean is going to talk to you more about that. Snapshots and clones, these are really efficient and you can actually do them with NetApps cluster data on top today. They typically take less than a second to complete. At NetApp we say that they are space and time efficient if you choose to go with that back end. So consistency groups is a really cool feature that we have in the Liberty release. So if you have a typically multi, if you have a typical multi tiered application and you want to take snapshots of that application that are consistent, you can now do that with consistency groups. So I'll now pass it over to Tom who's going to take us through some new features in Liberty. Thanks Akshay. So in Liberty we implemented a lot of different features. Here's a word cloud with all the features and we are now going through a couple of them. First thing we did is we implemented something called share instances which is a feature which is needed for share migrations and replication. So this is only facing the administrator so end users don't see anything from that. The reason why we integrated that is that for example when you migrate a share, you are doing that in the future with share instances and the share which is visible for the end user doesn't change. So it's basically about the UID in the share which has not changed even if you migrate or use another application for that share. Next one is REST API micro versions. I think that was implemented first in Nova during the kilo cycle and Manila adapted it and I think other projects are also looking into adapting that so there was a session this morning for Cinder and so they are at least thinking about it to do that as well. The thing here is that we can now evolve the API incrementally and theoretically we can also now break the API without breaking the user experience and how it works is that the client just sends an HTTP header with a version he wants to use and the server returns the correct blob of data. Next one is experimental APIs. So that's also implemented with an extra header so an HTTP header and the thing with experimental APIs is that they can change at any time or we can remove it without any warning. So I mean we don't want to do that so we want to design the APIs carefully but yeah it's an experimental API and if you want to use it you have to send this header and in the following slide when you see this experimental icon on the slide that's an experimental API. Next feature we implemented is extend and shrink so you can now extend and shrink shares via the command line interface or the horizon dashboard. It's not implemented in all drivers but there is a documentation so the link at the bottom of the slides point you to the documentation where you can have a look which drivers implement this feature. Next one, consistency groups. So with consistency groups you can group together different shares and that's very useful if you for example have a data of a database on one share and for example application data or log files on another share and you want to do a snapshot and you need to be sure that when you do that snapshot the data of both shares is consistent and that can be done with consistency groups. So the actions you can do with consistency groups is basically you create a group, add shares then you can create a snapshot from this consistency group and you can create a new consistency group from a snapshot. Next one is external CI for all drivers so that was a huge thing for all the vendors so thanks for doing that work we have now CI systems for all different drivers in Manila so we had to remove one driver at least but it may come back when they implement the external CI system so thanks again to the vendors. Next one is oversubscription that's an easy one so you can now oversubscribe your available capacity the only thing you need is thin provisioning in the back end and with that I hand over to Jean. Thank you Tom. Alright, as you can see we had a very busy Liberty cycle and I'm going to speed up the pace with some of the cool features that we actually added to Manila this cycle. The first one I'm going to touch is actually share migrations to those of you who know Cinder the block storage this is a very important capability to have when you do a back end so to those of you who need to actually migrate the data from one back end to another this is how you do it. As Tom indicated this is an experimental API so we do want you to start test it and play with it but don't use it yet in production actually we're going to have another cycle in Mitaka to do a lot of the additional work ahead the work right now doesn't include the retyping so if you have a changing of protocol between back end to a back end it's not yet supported. So how does the basic implementation works to those of you who know Cinder Cinder does the idea when you do share migrations you want to leverage the back end capabilities when you do the migrations it would be faster and the basic implementation if you don't use any back end capabilities will be basically using R-Sync to do it and to those of you who have experience with R-Sync you know the limitations of that in Cinder for example the fallback is using DD operation right so it's pretty similar another important milestone is the support for availability zones how many of you use availability zones today in OpenStack in regions just raise your hand this is an important stuff so we talked about migration as one use case we're going to talk about replication as another use case we need to be able to work with OpenStack Nova alignment around regions availability zones we can actually do operation between different availability zones so this is very important when you do migrations as well as replication and now we're going to talk some of the cool things we've actually done this cycle and I'm glad to say that this is an effort we actually completed with Manila and in Red Hat specifically thanks to colleagues of mine, Trevor and Ethan that actually drove this innovation we actually had them work both on Manila side and on Sahara to make use of the all goods that Manila brings to the table and the main use case we're actually looking to do with the shared file systems is to store the binaries of the jobs so when you do jobs for your Hadoop or Spark in elastic data processing the NFS is actually very ideal for that purpose so another point in Manila is today you have to do the plumbing work to mount the shares when you create manually at the guest level so what we've done is not just be able to add the additional drivers such as NFS and actually in the future will be others like Luster Press, FFS etc so we get all the variety of file systems for data processing but also deal with the automatic mounting so Sahara now is when you mount the we'll do the mounting for you as part of the EDP process if you're running the EDP as a user the mount will be happening automatically and this is already in liberty so to those of you who have any big data on your radar please use this and we like to get your feedbacks as well so you can mount NFS shares API and another work we've done is actually Manila provision HDFS so HDFS is one of the file system that Manila supports not out of the box so till now we had to use Swift in Sahara now we can actually use HDFS which is the native Hadoop file system out of the box with Manila and this is pretty much how it looks so on the right we have the data sources working on the Manila provision HDFS one limitation right now is the security this is still a single user so it's a non secured operation but this is still on our radar to deal in the next releases so now you can actually run Sahara directly with Manila using the HDFS out of the box and the other use as I mentioned is of course the use of NFS shares so in this example actually using Gloucester nodes as a back ends for exporting NFS shares to Sahara moving on very hot topic containers so at Red Hat we started to take a stab at actually what would it take to provision storage to Ceph and CephFS by the way has now Manila driver so being worked at so that's something coming up so those of you using Ceph will be happy to hear that CephFS is going to be production ready and Q1 2016 which is pretty much next quarter and we started to work with CephFS and Manila and we wanted to see how what will be the options of actually exposing the shares to our containers and there's several approaches actually that we end up so first of all in general using Manila exposing shares into containers is much easier than doing the plumbing work in block right because you can simply bind a share into the container in a space simply as that however it doesn't end there because we need to deal with security we need to deal with performance we need to deal with the whole life cycle of attaching and detaching the share to the container this is why we started to look at different approaches of actually doing it and we started mounting as I said the CephFS in different options and we're now looking at using Virtafas to do the plug-in work so we're looking at different options and now we actually mapped another use using Vsox Vsox is actually a protocol introduced by VMware and we believe we can actually use here it simplifies the work with containers because it introduced the Manila share as a black box to containers so it's both benefits from performance and the security aspects there is further work of course and Nova and there's actually design sessions currently going and I think last hour was specifically on this topic so keep it under the radar it's more than used to come we're doing all the experiments right now and with that I've showed you everything we just did in this release we want to give you a taste and how we're going to use it so great, we've done all this work we want to start playing with Manila give us Manila with the reddit one Tom? I'm going to show you a short video about the integration of Manila in the current SUSE version which is open stack cloud 5 so Manila is the attack preview and in the next to the version it will be fully supported so the controller is HA and we're supporting NetApp drivers and you can use other drivers as well with a custom backend so for deployment we use crowbar which is based on Chef so crowbar uses something like barclamps to deploy a single component of open stack so I'm having a ready cluster and just showing two minutes how to install Manila and how it works so here we go, that's the overview we have a couple of nodes we have two clusters, one for the services and one for the network now we have all the services already installed, we are creating a new so called barclamp for Manila so Manila can have multiple backends in parallel and that's also supported in crowbar so we are creating a new backend here for NetApp with a single virtual machine storage backend so adding some parameters so there is a default backend and here is an example how to add custom drivers, so you basically give it a name and then you can use the driver class here and add key value pairs which are added to the configuration file so to Manila.conf but we are not using that here, I'll just remove it again and you assign the rows to the nodes so we are going to remove the default assignment and now we are just using the services cluster which is based on pacemaker and at the Manila controller there and the Manila share services are added to two different nodes and then we deploy it, I think I removed the last, so the next two minutes because we don't want to see or we don't want to wait two minutes for nothing and that's the user interface, so Manila is also integrated in Horizon, there is an extra project which is called Manila UI that's integrated, we create a share type here which is needed before you can create a share itself and then I think we create a share and that's it basically, so it's just about integration what's currently available so creating the share and that's it so with that I give over to Sean again for the write-out demo thank you Tom so moving to RELL-OSP RELL-OSP 7 is actually let's see how I can get out of this first sorry, so let's start the demo so we've integrated Manila RELL-OSP 7 Manila is a tech preview still and we also integrated as you can see the UI plugin that Tom mentioned that was introducing the KiloCycle upstream so here we're basically going to create a share type, by the way whatever you're seeing here is actually already supported in the new OSP director, we have the new deployment tool in RELL OpenStack 7 and both NetApp and Gluster are supported out of the box so we're basically with director which is a big news for the customers so now you can start with it so basically what we do is create a share type so I can create a share type for NetApp driver, for Gluster driver HDFS use case is one of them that you can see and then I go and simply create a share in this case I'm creating a WordPress share giving it description showcasing Gluster FS but if I was doing a Sahara workload I would basically choose HDFS for example or NFS if I want to use it for the job binaries that I showed earlier we need to map the relevant size in this case it's the Gluster type so I need to match the size that I actually provisioned on the Gluster server and just give it the relevant metadata information or decide if I want to mark it as visible for example and then simply just mark, simply as that so the Manila UI is pretty much very easy and intuitive as you can see across the board and as soon as the share is created you can go and look at the list of share created and there's a link you can open the link and see the export information of the share and then basically use that to export it to the Nova instance so pretty straightforward UI and of course everything we're going to show you is a client as well right in CLI so that's pretty much it all right and with that after we see what's available today and as you can see we showcased a lot of features that are already available during the Liberty cycle by the way last point regarding the Reddit open stack platform 8 is going to actually have a full support for Manila out of the box and one of the slide actually I missed is just to talk about this one real quickly and we have done tight integration as I mentioned with the deployment tool so out of the box you can select the relevant cluster and net up drivers and one of the things we're doing to get Manila production ready for customers is we're going to initiate the certification program so all the vendors in the crowd now you can the beta is coming up actually next month and you can start working and get your plug in as you see you have right now 14 drivers already for Manila and our work is actually to make sure that when you plug Manila to your real open stack it will be good to go all right so let's jump to Mitaka Mitaka was a very as you know we have a very intensive design sessions and a lot of the design sessions are actually completed and now we can give you a quick look of what's coming up as I mentioned earlier one of the big things on the plate for us is to complete the cycle meaning we need to deal with the mount automation itself so when you create a mount you can automatically mount it on the gas and that's something we want to solve rolling upgrades is a big thing in open stack in general we've lost the board effort in all of the services right now to get to a more seamless upgrade process we're using version objects and Manila is of course is part of this effort and this effort is going in Cinder Innova etc and of course we want to get open stack more robust more work around export location metadata and one big item again on the plate actually we have two big items here in the list Manila quality of service and the share replication that we're going to demonstrate if you look at the Cinder matrix for everything you can do in the driver so the only thing we're missing right now after we got the migration in is quality of service and replication and believe it or not Manila and Cinder has a feature priority alignment already but it's like Manila's steroids right we really got far so what's Manila quality of service in two words those of you know quality of service in other areas by the way Neutron now in Liberty supports quality of service as well it's basically allowing the Manila admin to create service levels around IOPS, bandwidth, latency right because it's one thing to get great here's your share on demand it's cloud fashion use it but once you have like thousands of shares and you have a lot of shares going out of control maybe you need to start managing your share and adding quality of service here and this is how we actually deal with this problem share migration loose ends we had a lot of more work to do because we found issues in the as you know the code is still experimental and retyping for example is not there yet and the next thing on the plate is actually shore application and before we just hand it over to my colleague actually I just want to note that the API is now here and it's experimental and we're actually calling vendors to start and getting drivers so we can get shore application going and of course the biggest thing in terms of use case for shore application is disaster recovery right we want to be able to have replication between back in the back end and between zones as you would be discussed all right by the way we'll be here later at the end so we have questions regarding what's coming up and the things we've done in Liberty come and ask us questions I'm not sure we're going to have time after the demo all right let's go thank you Sean so I'm going to talk to you about the state of the art feature called share application this is coming up in Mitaka the code is already pushed upstream so you can go ahead and check out the upstream code on github if you would like and see the details there one of the things that enterprises and even small businesses are looking for is non disruptive operations and the way you achieve that is through high availability if you wanted to achieve high availability within an availability zone and Sean touched on availability zones earlier it's basically a data center you can think of an availability zone as a data center if you want to achieve a high availability within a data center the way you do that is you provide clustering of your storage controllers you can do that with cluster data on tap today net apps cluster data on top today you can cluster the storage controllers together and if one of the storage controllers where to fail maybe loss of power loss of network connectivity then it fails over to the other storage controller but this actually leads to the next question what if the whole data center goes down maybe due to loss of power in that case we need to have that's where share replication comes in to provide high availability let me now take you through a demo for this so let's take a look at the third demo we have so we're showcasing Manila disaster recovery over here available in mid 2016 we have two availability zones and there are no Manila shares right now we don't have any Manila replicas right now either share replicas right now either we have implemented this extra spec called replication and I'm just showing that extra spec over here extra specification so what we have done is we have listed the share replicas we have created a share and listed the share replicas we had one replica in that case at this point in time we are looking at the cluster data on app through NetApp's on command system manager and just making sure that the share IDs do match now if you have thousands of shares it's important to match your share IDs appropriately and that's what this is showing so now that we have matched the share IDs just for verification we're going back and listing the share we are going to allow access to the share because each share that you create in Manila has to be specified with an access list to allow access so we just did that we are creating a directory to mount the share and then we are going to mount it this is an NFS share so we mounted with the NFS protocol so we're just making sure that the share is mounted at this point in time it looks like it is mounted so we're placing a couple of files in the share and the first file is a text file that just says this is some content and the second file is a 100MB zeroes file so let's now actually get into the actual feature which is we are creating a replica in the second availability zone and what we do that using the command share replica create so let's go back and list the different share replicas and at this point in time we see that we have two different share replicas one in the first availability zone and one in the second one so we're just waiting until the second replica is in sync with the first replica switching back again to make sure that in this case the netapp storage backend share IDs do match you don't necessarily need to do this in your environment but if you have a very large deployment it's a good idea to check with your storage backend now that we have verified the two shares the share and its replica we can go ahead and actually promote the share replica and make sure that the data in the second replica is consistent and we have just promoted it let's go ahead and list the share replicas and let's wait for them to get back in sync so it takes a couple of seconds for that to happen so we see that there's been a replication change we see that the first replica is out of sync and then it's going to go back to in sync at this point in time the two shares are in sync and we can go back and mount the share replica in the second availability zone and make sure that the data is still there so we see that the share is mounted and if we just do a directory listing we see that both the files are in the share so that concludes the demo so here's how you get involved with Manila all the resources are over here available on github and the wiki and launchpad if you want to also contact us feel free to get on IRC on the channel that is mentioned over here we have weekly meetings on Thursday at 1500 UTC the websites for NetApp, Red Hat and SUSE OpenStack are mentioned here if you want any information on NetApp in particular it's http netapp.github.io Manila related sessions there have been a few sessions that have been completed already please go ahead and watch them on YouTube if they are of interest to you there is a meet up tomorrow that is happening and of course feel free to attend that as well so at this time I would like to invite Tom and Sean to come over to take any questions we'd like to thank you for staying with us till the end of the day on Thursday please do submit your feedback through the tool through the iPhone app or android app which ever platform you use thanks for staying with us any questions you have we're ready to take them yes just a second so when I create a share from one tenant can I use this same share from another one the beauty of you want to take that? so Manila actually allows you to do multi tenancy as well that's one of the big use cases that we actually showcase so the quick answer is yes but again you need to specify the tenants permissions right so if you were to provision an NFS share what you can make sure is you can set the access policy for the network to be sufficiently to allow access to the tenant network on the second tenant that you're interested to have access to the share and that would at the networking level at the networking level I have to share to give them access rule for that share thank you and Manila you will have to do a Manila create to create the share and then you'll have to do I think it's an access allow to allow access to that share alright yes can you pass the mic as far as I understand what you described is how to create a share how do I actually mount it inside of a VM is that automatic as well do I define the mount point or do I have to log into the VM or have to configure the FS tab as I mentioned earlier right now you have to log in and do the mount but one of the features we're working for Mitaka is actually to do the mount automation as I also mentioned this is already supported now with Sahara so Sahara does the work for you automatically out of the box so you mentioned one driver that had been dropped I think that's the GPFS driver is that right is that dropped for liberty entirely or there was a blocker on you saying it was failing CI or something like that so there wasn't external CI for the GPFS driver but so now it's added back so that's not the driver we dropped GPFS is in liberty GPFS is in liberty yes Sean it's a red hat question so with secured multi-tenancy on containers you touched upon it should work but you also talked about tech preview currently so which open stack version will work with which red hat version if I work for Veritas and we are working on a manila driver which is not part of those 14 I'm waiting on CI CD but once we get there how do we make it work with manila on red hat with which what is the right combination so first of all as I mentioned manila is tech preview in rel sb7 so probably by the time you get rel sb8 this is coming up which is going to be fully supported and as I mentioned the beta should be soon available for rel sb8 so rel sb8 with red hat 7.1 okay hey I've got a question I'm running netapp7 mode can I use the manila driver so that I have to get back to you so most of our work is on cluster data on top is that right it's cluster data on top would be no 7 mode it's not right now okay thanks probably not more questions that was asked earlier does their share replication already supports consistency groups the answer is not yet but yes it's something that we have on the radar and of course the main use case when you create the replication for applications like workload some of the workload sites such databases need are going to be split like database data logs etc on different shares and when you create your replication you pretty much want to make consistent replication that you can mount on the other side in case of failure so this is why consistency groups are important in the context of replication and this is something we have on the plate with that I want to thank Akshay and Tom and all of you actually staying that late and enjoy Tokyo and the rest of the summit to those of you who are saying thank you very much