 So good evening everyone. We hope you had a great summit and we hope you enjoy This presentation as well We are here today myself. My name is Akshay Parthasarathy. I'm a technical marketing engineer at NetApp We're here today with a dupe with Greg Elkenbard from Marantis and Tom Beck told from Suza Greg and Tom would you like to introduce yourselves? Hi, I'm Greg Elkenbard. I'm senior technical director from Marantis. I OpenStack architect and I focus on networking and storage Yeah, I'm Thomas Bechtald. I'm working as an engineer for Suza on the cloud product Thank you So let's Take a look at the agenda. This session is going to focus on OpenStack Manila, which is the file share services project and the first thing We would have for you is the introduction and the use cases for Manila Followed by the current status of the program and the Newton release key contributors to the program which which includes NetApp Marantis and Suza and Then we'll take you through two of the major features that were released in this Newton cycle The first one is share application the UI part of it in horizon Which is available through the Manila UI plug-in and then we will also take you through the share migration demo Share migration has been enhanced in this release Greg will then walk you through Manila installation and fuel and Tom will take you through Manila installation in Suza So Manila is for shared file systems. What's in there is for black block storage What you can do with Cinder and dispense block storage volumes you can do with Manila and dispense file shares be it Windows file shares SIFs file shares or UNIX like file shares UNIX and UNIX like file shares NFS You can also work with HDFS data using Manila It supports a wide range of network topologies and these include neutron networks the legacy Nova network Using their dedicated plugins and you can also use standalone networks So in the figure there I show you two Manila shares one of the shares is for the R&D department It's just named R&D and What I have done is I have enabled access to two of the VMs in your open-sat compute cloud and Likewise, we have another sale share just for example and you can enable access to a couple of other shares So that in a sense what I'm trying to say is you have a shared file system that is accessible by multiple clients in Manila So I do want to before I skip to this list I do want to point out that just like you can have open-sat compute instances access your shared file systems You can also have containers non-Ompa stack VMs or bare metal nodes and also open-sat bare metal nodes. So it's not restricted to open-sat compute So in terms of the user foundation Open-sat foundation results as far as production deployments We see a lot of adoption as far as NetApp is concerned also the generic driver which includes a generic Implementation on a Linux server, which is a Nova VM. So there are also a number of other options available for you as you can see there So the use cases for Manila are numerous and these are just an example. So maybe you have databases such as SAP or Oracle these that we have seen with customers or maybe you have a standalone shared file server, right? You have an admin in your organization and what they're doing is they have implemented some custom Perl scripts or You know, they're just doing manual configuration for the file server What Manila can do for you is by using the open standard REST API. It can serve to dispense such file shares in a much more In a much more efficient way So you could also make use of Manila for things like DevOps I'm going to show you a demo in which we have used Manila for the continuous integration logs So I'll show you that demo and then you can also use it in terms of hybrid clouds and private clouds These are just examples So one of the value ads of Manila is that it allows you to create your own marketplace or what we call the storage service catalog if you If you tear your storage for example, maybe you have a good best and better tier you can have what are known as extra specs and Leverage these extra specs Specifically for each tier in this case. We have a Dev and test workload or use case And we're using it with certain characteristics that include compression quality of service and so on so Likewise, we have a database as a service Workload we can use another tier for that and leverage a different set of extra specs So VDI in this case is another great use case and we are using a different set of extra specs so these kind of The marketplace each of the tiers in the marketplace are not restricted in the number of workloads that you can have so maybe you Just like how you have the database as a service workload We have provisioned again and an NF free workload that also makes use of the best here And it's not always good better and best you can customize it however you want in this case We have another tier called analytics and this is for your analytics workload In terms of the status as of the Newton release Manila has been production ready since Liberty and In Newton and in the metaka cycle and more so in the Newton cycle. We're enhancing that The project further the total number of drivers which includes an app of course is 17 And we have completed a total of 14 blueprints in this cycle You can see the number of bugs resolved In terms of the lines of code It's over a hundred thousand lines of code and the two major features are the Manila UI integration And this is for share migration and share replication. I will show you using the Manila UI which share Replication so this will be a multi availability zone demo and then there's also the two-phase share migration Which can significantly reduce your downtime Also already included for you right you have things like replication using the CLI as of the previous release itself and then consistency groups to take consistent snapshots of multiple Manila shares at the same time and flexible share sizes This is a depending on the back end you use you can increase or decrease your share size on demand You can go from one to a hundred GB and then go back to a 50 GB if you choose to do that Things like importing new file shares into Manila be it NFS or cifs The major players you can see them here NetApp is one of the contributors you can see all the others as well The project team leaders actually an architect at NetApp We also have representation at Maranta Sousa and a number of other companies And I have talked to you about the lines of code already. So we're a hundred thousand lines So I want to show you both of these features the first one being the share replication UI Through demos. So before I show you the demo, I'll give you a brief overview of what you're going to see This is your Manila share and you have some logs that you have retrieved from your continuous integration system They can be any type of logs, but in this case we have used that We use log stash to generate event data. So we run a CLI command with log stash to generate this After that you can use elastic search to generate indices on the event data that logs stash generates for you Using the indices What we can do is we can use a visualization tool such as Kibana to give you that kind of graphical Analysis that you can use for logs. So this can be very productive in terms of if you have an IT department With huge amounts of logs to go through What I've done is I've replicated the share from one availability zone to another and We're gonna show what happens when the first availability zone goes down and what The way we simulate this is I simply go and disable the NFS service on that particular share So as a result of disabling the NFS service, you're going to lose that visualization that you see on Kibana and then we promote the second share to the primary and We generate the event data we get the indices from elastic search and then We again get the visual visualization that we want through Kibana So at a very high level what we're showing is the visualization through Kibana once The visualization through Kibana after a disaster has occurred in the first availability zone So I'm logging into horizon. This is everything's going to be done in the UI and We create a share The shares 50 gigabytes in size and DR enabled which means disaster recovery enabled and the availability zone We choose is Barcelona There is a second availability zone called Madrid and we'll be taking a look at setting up a replica of the share as the demo proceeds So for now what we're doing is we're adding a rule and We're going to give it a Manila by default denies access. So we're explicitly allowing access to the lock stash VM So that is the export policy and we can use this export policy of the Manila share for lock stash So here's the lock stash VM and we are mounting using the export policy and You can see over here it is mounted We're going to go into the mount point and we're going to make a couple of directories The first directory will be your CI logs and the second directory is the indices So the CI logs we we what I've done is have pulled the logs from the CI system locally At net up and I'm using that as inputs here. So these this is the configuration for lock stash just Showing you that you're using this particular directory for the log stash logs and Then we're using elastic search as the search engine Now if we look at the elastic search configuration what we'll find is the elastic search is using the same mount point But the directory slash log slash indices. So it's a new directory within the same mount point I'm starting the last week service and then over here. I start the log stash Service so that I can generate event data that elastic search can then run on so can generate its indices So this is the log stash running elastic search has already been started so what we're doing is we assume a few minutes have passed and We are looking at Kibana, which is a visualization tool and we should be You know after these few minutes that have passed be able to you know Search for errors and error messages and see if we get any search results We're just searching for the word error Looks like that has a few matches and then we search for error and no valid host looks like it still works fine So essentially it's working fine in this Barcelona availability zone So this is the Manila UI and what I'm doing is I'm creating a replica in the Madrid availability zone So we choose Madrid We create a replica We made it we wait a few minutes So I have fast forwarded the video over here because it is a huge share But we wait until it becomes available and in sync and this is the actual disaster I'm disabling NFS on the primary copy, which is the Barcelona availability zone And at this point in time the status goes right, so I have disabled access doesn't have the data It goes right. I promote the replica in the Madrid availability zone just takes a few seconds for the promotion to happen and now we get the The export location of the Madrid availability zone Manila share So we are unmounting the Barcelona availability zone share that is no longer active And then we're mounting the Madrid availability zone share Going to check whether we have the data that we need So we go into MNT logs and we should see that CI logs and indices folder So if we go into CI logs, do we have the data that was Replicated so we have a number of text files. These are basically the logs that log stash is running on So what I'm going to do over here is I'm going to run log stash again to generate more event data on this With using the new Madrid availability zone and this is going to you know, basically keep running and In parallel we have the indices already from the Barcelona availability zone So we're going to just take a look at Kibana and see whether we can see those search results again So if we see that Kibana is up it's not in a status read anymore and we can see that we are able to search info share and created those are the terms We have used seems to be working fine so That's basically share application in this particular CI CD environment So the other thing I want to show you is Manila to phase migration and I do have a demo for this as well So for to phase migration what we've done is let's say you have a hundred terabytes share with 95 terabytes of data So in the in the previous iteration, what would have happened when you migrate is that It would have copied all the data over and until all the data is copied You actually wouldn't have access to the Manila share that you are migrating from In to phase migration We're actually giving you read-only access as the data is being copied and then maybe at a schedule downtime You can quickly switch over to the new share Migrate the share over and your end users can use the new export location So significant reduction in terms of the downtime The Stuff we will be working on is the loss of metadata in the case of netapp the metadata loss is your snapshots But these will be fixed in the future in your future. So we will be adding the ability to have writable copies So even as you add Even as you're migrating you will have the ability to write Preservation of metadata including snapshots A non-destructive migration. So this is interesting non-destructive migration. This is the demo. I'll show you I was already checked in by a colleague of mine Gotham Right after the Newton release and this is the demo. I'll show you but as of the Newton code This has not made it there And we have driver-assisted migration So if you have that large hundred terabyte share you want to take advantage of the back end and do the migration As efficiently as possible. So that's also coming up for you So on a timeline with the current two-phase migration Your client is accessing the Manila share the administrator Issues a migration start command the client has the ability to use the share in a read-only mode as the migration is happening and then after After the after some time the shares going to be completely all the data is going to be completely copied over and then at that time If you query the open-sac CLI it's going to indicate for example phase one down The administrator issues a migration complete command. Maybe this is during a shed schedule downtime and Then at that point in time the original share is deleted the new share can be accessed by your client So this is the demo. I do want to point out that this is using non-disruptive migration Which is actually checked in? But it's pretty cool in terms of what it can do. This will be part of the Ocata release So we're creating a Manila share and this share is actually used for multiple center volumes It can be used for center volumes. You're going to show you an example of using Manila for sender We have a 200 gigabyte Manila share and what I'm doing over here is I am I'm allowing access to this particular sub-nap For that Manila share So we have the export location here and using the export location what we can do is we can use this for sender So I go into I go into sender conf just to show you the back end that we're using we're using an adab back end in this case but you can use another back end that supports NFS and We go into shares dot conf. So what I'm doing in shares dot conf is I'm actually Pasting the export location of the Manila share, right? So and We restart the sender services and we should see that You know the sender service starts up successfully. So we wait a couple of seconds So the first thing we're doing is we're actually getting the pools and what we notice is that okay? There is one active pool. It looks like it's in that 172 2124 sub-nap and it has the same export location that we specified earlier So if you look at the services now, we see that we do have that center volume services up and enabled enabled and up and This is actually where we actually create a nova instance that uses that is backed by sender But sender in turn is using that Manila share So here's our Nova instance that was created take take a few seconds to actually go into running mode Okay, and here's where we split into multiple screens and at the bottom screen What I'm doing is I'm showing you The Nova instance that has booted up and that is its IP address So I'm going to log into this Nova instance from the top two screens So that is the first screen that I've logged into and then we log into the same Nova instance from the second screen okay, so The point of non-disruptive migration is that it has to be truly non-disruptive So what we're doing is we are actively going to generate IU as the migration Is kicked off and completed So what we notice is that this? particular Manila share is on this back end called Open stack to which is an AG gr2 the source is a GG are to that long string that you see there The destination we do have another destination that is available, which is a GG are one ends with the word a GG are one So I'm going to actively generate IU as We go as we migrate the Manila share from a GG are to to a GG are one and the way We actively generate Ios by using DD and then on the other screen We are going to actually look at the IU and it looks like we have around 400 IOPS or so So if the disrupt if the migration was disruptive that IU should basically go down to zero and Not only that we should lose connection to the instance, so We are going to perform the migration To a GG are one and we're looking at the progress here So after some time the progress is going to say migration phase one done and After that has happened We do see that the Ios still seem to be going on and we can issue the migration complete command to actually complete the migration So throughout this whole process, right even after migration underscore success We see which will happen very shortly Just waiting a couple of seconds So migration underscore success we see that you know the Ios are still going on We have around 400 Ios or so there's been no loss in connectivity to the instance So truly non-disruptive So that's everything I have for you We had the share application UI integrations and the migration feature through the two-phase migration That was introduced in the metaco cycle I now want to pass it over to Greg to tell you more about Manila and Mirantis. Thank you so much All right, so Mirantis seven participating in Manila project for a while now But before open stack 8.0 There was no Automated way to install Manila if you needed Manila you could always add it as an extra service to an existing running cluster but it did require manual installation configuration in fuel 9.0 and Above we've added installation and configuration automation So now you can automatically install Manila whether when you create a new environment Or you can go back to an existing 9.0 environment and add Manila to it So a little bit of detail about the fuel plug-in. It's an optional services Just like all of the other services such as Murano and Sahara just to keep the footprint small And then you have ability to control placement So the Manila API and Manila scheduler are hardwired to go on the controller because their footprint is relatively small But Manila share and Manila data are coded as separate roles Which means that you can have a flexible placement either on the controller or on another standalone node in order to Provide it a better service The initial release of the fuel plug-in itself that Manila just a plug-in supports the generic in the NetApp driver And if you need additional drivers, these can be added afterwards so let's go ahead and Go to the demo all right if you guys can see that what we're what we're trying to do is Install Manila as a plug-in some manual some CLI configuration obviously is required in order to load a plug-in into fuel and We're going to use fuel plug-ins install command to install the plug-in as you see now we have a plug-in installed so we can go ahead and Go to the fuel UI and create the environment Basic environment creation step standard for any fuel-based deployments just define the name What OS is running on the VLAN the networking type so now here is Manila As a service you can enable it in your existing environment the same would work if you had to add it to another environment Any open-stack environment requires controls and compute nodes which we're defining right now and we're going to go ahead and Add two more roles as you see right now for Manila share and Manila data Next we're just going to do a quick networking test to make sure that we can pass the network traffic because we reshuffle the interfaces and After that test we'll just take a look at that everything is fine and deploy the environment which will create the environment So in the settings as you can see we can use the generic driver We can also add support for the NetApp driver here We are configuring some basic configuration information that's necessary for us to access the NetApp driver Both drivers can be activated at once in the environment. It will support multiple backends Up network connectivity check is successful So let's go ahead and deploy the changes through the magic of cinema Obviously, we'll accelerate the 10 minutes to 30 minute installation process for open stack So now you see that the open stack has successfully deployed will run a quick health check Obviously since this is running in a lab Some of the tests won't won't pass but it'll just demonstrate the fact that the environment is up and operational and then we're gonna go Do some IO? All right, so we'll go ahead and launch horizon Create a couple of shares and write some data. So we're gonna go ahead create the share share name Basic information is required and you can supply Extra specs to specify the back end that you want to do in multi back-end use case Okay, so let's create a share on that back end. All right, awesome. Okay, so the shares created Now we will want to Create another one, okay and After that we're going to go create a instance and We will mount the share on the instance and just demonstrate that the IO can be done to it All right, so the instances are running the shares are available. So now we're going to go Into the instance UI and make sure that the shares are Accessible Obviously because the default configuration on Manila Does not specify any access controls the initial mount command will fail And you'll see that the Manila access controls are actually working And then we're going to go add ourselves officially to the access control list try the mount command again It will succeed so we're gonna go back add ourselves to the access control list and try the mount command Now the mount command will succeed and we'll be able to mount a share And write some data on it great So now we're gonna go into the second instance and make sure that this is truly shared file system So we're going to try to read the share from From that instance and make sure we can Write to it and append to the file system as you can see we're able to read the share written by something else So let's go ahead and echo into it to Add another line to it make sure we can also write from the second instance Excellent, we were able to append the file. So obviously both nodes can access the file read, right? All right, so this concludes our basic demo and I believe Next is the Susie integration with Manila. Can you hear me? No. Oh, right now it works. Okay, so I'll talk a bit about the State in open Susie open stack cloud. So the Manila state Well, so Manila is fully supported and integrated since Susie open stack cloud six So the controller side of Manila which includes the API and the scheduler is highly available and we are including the NetApp driver and The generic driver for testing basically so that's not supported But you can also add any custom driver or whatever you want For the upcoming Susie open stack cloud seven there will be the seph of s driver also included And of course, it's updated to the to the actual Newton release So, um, yeah, I'm giving now just a quick demo how it looks like so Susie season crowbar for the deployment and so that's a pre-deployed Yeah, environment with a couple of nodes. We have a cluster running or basically two clusters a service cluster and a network Cluster running and we are going to deploy Manila now. Oh No, not like that. Oh That does the trick. Okay So here we have the overview of the two clusters So service cluster and network cluster These are a list of services we can deploy. So there is a lot of stuff already deployed and we are going to deploy Manila now You can configure multiple back ends that's similar to to sender and we are configuring the net net app driver now and Just create a new back end at the V server name and also host name and port and stuff you need So that takes a bit So here we go So then there is a default back end pre-configured We are going to delete that because we need we don't need it yet And here's just I'm just showing how to configure any other driver which is available in Manila So you just add the driver class there and you can add Random key value pairs which end up in the Manila con in the end So you can can enable any back end you want or any third-party driver if you have one So we are not using that there so we just have the net app back end and Here we are assigning the roles to the different nodes or clusters So we are going to use the service cluster for the Manila service role which includes API and scheduler And we are deploying the Manila share service on two different nodes Then it takes some time. I think I cut that off. Yes Yeah, now we are going to the to the dashboard to a horizon and Try to create a share So first thing you need for a share is a shared type where we can where we can Specify extra specs for example that actually I mentioned already Previously and afterwards When you created the share type you can create a shell and select the share type So you give it a name you select size and the share type and you create it That's it So if you have any questions about Manila, there are a couple of resources So there is of course the source code on github and there's a wiki page Which links to different resources. There's ISE channel and there's a weekly meeting on ISE Which is also on the wiki page. So there are also resources about net app. We run this in Susie if you're interested in that Yeah, and thanks a lot. So Any questions? Sure replication from self to not apps Okay, so let's take that offline. I I have not personally tested that I'm pretty sure you can't so Replication just works in one driver and not all driver supports a replication. I'm sure yeah I'm sure as the project is enhanced. Yeah Any other questions? Great. Well, we'll still stick around if you have any follow-up questions. Otherwise, have a wonderful night. Thank you