 I'm Brenda McLaren. I've been working in the storage stack for, oh, I remember that I wanted to admit, but I just joined Red Hat not quite a year ago, so I have a year coming up in September. Great opportunity to learn stuff. And then the integration with OpenStrike and OpenShift. So what I learned to do was kind of cover deploying stuff externally and then integrating with OpenStrike and OpenShift and what's required. So I'll go over stuff installation, I'll go over the different components and the networks that stuff uses, some information about the pools, the data protection, and then security. And then we'll cover the Once we get that installed, I'll kind of show you the integration with OpenStrike and then the OpenShift integration as well. We have time, which I doubt we'll have, I was also going to show you the ODF installation as well. So the three four components to set is like starting with the OSDs, the object storage demons. That's what is actually wording the data to the disk, right? So it's a one-on-one relationship, primarily with the OSDs to the actual disks that are in the storage nodes. There are times when you can use like a bigger OSD, let's say like if you have an NVMe and you have eight terabytes, you may partition that and go with two or four terabytes and then have two OSDs to that one. But for the most part, most of the installations are one OSD demon to a disk. And that demon, obviously, all of this runs in container. So whatever we say, like the service or the demon, we're talking about. The stuff mine or the monitor is what actually retains the, what we call the crush map. The crush map is, if I can remember, it's off the top of my head, control the replication under scalable hashing. So it retains the mapping so that any, whatever the client wants to talk to set, the client doesn't have to go look up any information. It's just going to be able to go and like determine exactly which OSD it needs to talk to and to be able to pull that data from the object store. So the monitors, there's a minimum of three. That's why SEPH always requires a minimum of three nodes because you can't deploy two monitor containers on the same note. They have to be on three separate nodes. So that's the minimum cluster size. And then you have the SEPH manager, which kind of runs alongside of the SEPH monitor. It collects the state information and it also supplies the APIs to be able to pull information from the stock cluster. Two additional services are the RGW service, which is for the RADOS gateway. That's the interface to be able to use S3 or SWIFT. And then also we have the MDS metadata service, which is going to be the service that takes care of all of the file system data whenever you're using file services from the SEPH cluster. Both RGW and the MDS are generally deployed after RGW can deploy when you deploy the SEPH cluster, but the MDS has to be deployed after the SEPH cluster is already up and running. You can deploy multiple RGW gateways. Most of the demons, you can only deploy one container service per server, but with RGW, you can deploy multiple RGW gateways on the same node, but MDS, you really should deploy a minimum of two. You just want active one standby. There is a way that you can configure active MDS servers now with the newer releases of SEPH. So we have all these different services, and when you go to deploy, you can use what we call co-location, which is running multiple demons on the same SEPH cluster, on the same SEPH nodes, and the OSDs, which is going to give you a lower TCO, so that you don't have to have one server for MDS, one server for RGW, another one for the different monitors to kind of co-locate these demons. So in this example, we have three nodes, and we can deploy an OSD on each of those nodes, each one of the monitors to give the minimum of three, and then you can have Grafana deployed on the first node. You can have RGW or SEPH. You don't want to deploy those demons on the same nodes. So in this, this is a minimal configuration. You're not going to have, you're not going to have like HA for your RGW or SEPH if you're trying to deploy both of those. Then you can get into like a four node configuration. Again, you know, just like spreading these out in this particular case, you know, RGWs look more highly available or MDS, you know, for SEPH-FS is just on one node. This is kind of like something that's interesting that Darren and I were talking about is that, you know, kind of my note on the side is the minimum storage cluster is three nodes. But if you have a replica of three, and we're going to get into replica in a minute, you really should have a minimum of four nodes. So it's always whatever your replica is, you should have that plus one more node. I know that in some of the, like the director deployed software, the HCI environment for OpenStat, they have like a minimum of three nodes. But in the SEPH documentation, it says that if you're going to have a replica of three, that Red Hat recommends four nodes. And the reason for that is that if you lose a node, that is going to be down for any length of time, it means it's going to have to rebalance that data. And you're only going to have two nodes. So you're really like at risk, right? Going on to the network piece of it that uses two, it can use one, but we recommend two. We recommend one for the front end. Yes. Sorry, back to, so what you're saying, if you select three nodes cluster, it's probably better idea to put two extra replications. Correct. Okay. Yeah. And as it says that, you know, like if you had, if you, if you're using a replica of two, then yeah, you can have three nodes. And that's, that's not an issue. But if you have a replica of three, you really should have four nodes. So with the public network, that's going to be the client traffic. So if it requires to come in, comes into the stuff cluster is going to come across the public network or the client facing network, but then stuff likes to have a backend network to be able to separate that replication data, the rebalancing data, the heartbeat data off onto a separate network. One thing to point out, like especially with like, for this purpose, is that whenever a client writes to an OSD and let's say that you have the replica of three, it's going to write that data to the first OSD and then that OSD is going to be responsible for replicating that data to the other two OSDs in the cluster. And that's going to happen on the backend network. When the backend network is saturated, it will overflow to the front end network. Any questions on the network? Yes. I've heard many names for these networks. Sometimes it's public network and sometimes it's the private network. Yeah, private cluster. These are like private and cluster network. They use those interchangeable, but that's the backend network for that for that replication and backfilling and this app service, you know, like cluster traffic. So when we talk about durability, right, because we have to expect the hardware to fail, right? It's not an exception to the rule. It's the rule. It's going to fail. So the replica, the default in Seth is three replicas. So we use the last storage, right? It's it's using react the amount of data that you're writing. You can use erasure coding pools and they've kind of expanded it like over the last iteration, I think like in five and then in six, there's like additional pools that you can now use erasure coding on. And so it puts that it puts like some CPU overhead on whatever you use the erasure coding. But again, it's that it's that seven OSD that is taking care of that erasure coding. And so for those of you that might not be familiar with erasure coding, is you can think of it as like software base rate, right? It's taking that data, it's chunking it up and that it's applying a parity against that data. And then writing, you know, like spreading that data out across, you know, whatever the erasure coding scheme is. And I want to say there's, I think there's three off the top of my head that you can use. It's like eight plus eight plus four, eight plus three, and four plus two. So the minimum is going to be like it's going to split that that's going to put that object into four parts and then apply to two sets of parity and then spread that out over six devices. So that actually reduces the amount of storage that you have to use. So a lot of times when people deploy SAP and they're using RGW for objects, they'll use they'll use erasure coding for those object pools. But for the index pool, it's required that you use replication. And that's because like for that index pool or the metadata pool, that the consoles will have to be so quick, they want the data just replicated as opposed to erasure coded. But you can use erasure coded pools for the the RVD pools and for the the CFFS data pools. But again, not the index pool. And if I'm throwing out too many acronyms, let me know because like I'm not like, I know it's hard to keep up with them. I always, I always am challenged by that. But RVD is the rados block devices. So that's the block service that you're getting as a default with with SAP. And whereas like RGW is the S3 service and then CFFS is the file. And can you can you mix it match kind of like can you put erasure coding for RGW the same cluster and then 3x or 2x replication of another pool? You can. The nice thing about SAP is it separates everything into pools. So you can whenever you create the pool, you're going to set up what the data durability on that pool is. So if you create and you can create an RVD pool and like if you just created a default one, it would be replicated. But let's say you want to have an RVD pool that doesn't need the performance and like that your normal RVD pool would be, you can create that as erasure coded. And then you can create an RGW pool that is replicated and you create another one that is erasure coded. And like actually specifically with RGW, if you're familiar with S3 and some of the different capabilities, like you can you can do multi-part uploads, the multi-part upload that bucket or that pool that is that is used for that has to be replicated. That's because it can't have that overhead of having to take that ingest that data and in the to the multiple parts and then put it all together and then shoot it back out to the erasure portable is what it would do. Any other questions? So for our medication, Seth uses what they call Seth Act, which it's it's I'm not real real real like in-depth knowledgeable on curve rolls, but they say that it's similar to curve rolls and that you don't get tickets but you get tokens. But the way that the Seth Act's authentication works is that Seth holds a key kind of I think it's more similar to like AWS S3 where it holds it holds your secret key and you know your secret key but the two are never exchanged in the conversation between you and your authentication with Seth because you're going to give Seth some information and it's going to be like you're going to give it like your user ID the pool and the object name and you're going to hash that with your key and then and then that goes to Seth and then Seth can decrypt that because it has your key as well. And so that how that's you know basically how it's working to do that authentication without having to transmit that secret key. Whenever users are created the authorization is is granted whenever you create that user and so all users with Seth have to have read access to the monitors to be able to get that crush map right so that they always know where where all the what we call the placement groups for the pools it has to be able to get that information so it knows which which pool and which OSD to talk to so you get read access to the monitor and then you would get read or read and or write access to the specific pools that you need to be able to access whether that's an RBD pools FFS pool or or RGW and then the RBW like the S3 and the SWIFT that is like a different set of authentication that is handled through the RGW gateway. Am I doing it on time? Okay so now we're getting into this up installation but one thing that like is really stressed when we're working with with different customers is really make sure that you know your workloads and what you're expecting out of Seth before you actually go to do the deployment right because I mean the nice thing about Seth is that it that it does separate everything into into the different pools but you also want to make sure that you use the right disks right you don't have to have all NVMe or all SSD drives right you can have some of the data sitting on spinning disks especially with your with your object store right it's always better to put your object data on spinning disks and use your RBDs and some of your Seth FFS you're like on your on your higher on your you know like your faster disks. So really what comes to and I I had the opportunity to actually like learn how to deploy Seth with one of my co-workers for a POC that we did and it just seemed like like wherever OpenStat goes to deploy it's just like so long and just like so labor intensive it seems like and to me this is like it's really simple to be able to deploy this as an external cluster. So it requires the Seth ADM Ansible package that you have to install on the on the actually the first node in the cluster or whatever your administrative node is going to be and then once you have that package installed you create off your host file and then you're going to run your preflight label and then that's going to go out and it's going to court like some required packages on each of the the nodes that you're that's going to create your cluster and then my the way that I did is I always create a root Seth directory and then in that root Seth directory I have like my initial config I have my registry login information so that you you can pass that registry information on the command line or you can just use a JSON file and that way you don't have to pass the password on the command line and then once you do that you create an initial cluster configuration file and then you can bootstrap the cluster. So with that I'm going to get out of here and go to yep we're going to go right here. So this I already have the Seth ADM Ansible package already installed so if we just go to install and we look we have I have that the host file already created so I have four nodes in my cluster and that administrative node is going to be the Seth 1 node. Can anybody see that okay or do we need to I can make it a little bit bigger? Is that better? Okay. So let me see here. That's not the right window I was looking for. So here I just have everything so I just have to paste because you do not honestly we have to type all that. So here we're going to go ahead and just just create or run the Free Fight Playbook and you have to pass it that extra virus and you have to say where the Seth origin is. So in this particular case I'm saying that it's custom because we're using our repository in our lab as opposed to going out to like to the red hat repository and this this is going to take really too long because it's going to skip some of the stuff because like I've already installed it and then kind of flew it away and it's followed again. So that part's done so you see that didn't take very long. Then what we're going to do is we're going to go we're going to go over to root step and here I have some of the files already done so like I'm going to show you this registry login file and it's like something quick and dirty here that you're just saying what registry you're going to use. You're going to have to pass it your username and then whatever your password is. So you want to make sure that you put the right permissions on this file obviously um so because I'm actually using my information I'm going to copy my uh my oops right that and then okay so then I wanted to show you the um so this is the initial configuration file so you're going to put your four different like in this case the four different hosts you're having to have in the cluster and then there's this um this service right here for the monitor I'm going to tell the placement it's going to be on set nodes one two and three um it only needs to be on three of the nodes it's minimum requirement yeah um it's the best to use like three five seven etc um and then here is is my information for the osd so I'm starting to go ahead and put the osd's on nodes one through four and then to also use um the device uh slash dev slash vdb yeah on the placement there if it's going to be all the nodes can you just leave that out and it'll do all the nodes you can't leave it out when you can you can see that it's actually like uh I don't know like what it was like a regular regular expander I could have said I could have just said like stop after probably yeah I just know you want that I would do all no you have to actually stay with the placement okay thanks and so here is going to be the can and whenever I pop it in here you'll be able to see a little bit better so this is going to be what's going to um actually do the installation so we're going to give it um we're going to get the monitor IP address so that's that that's just one of the monitors that um one of the nodes that has the like the monitor on it right so I said step one was going to be a monitor node so I just took that IP address and then the applies back is this initial configuration you can pass it what the initial dashboard password is going to be there's another option that you can say uh dash dash no password update which means that you can set what the initial dashboard password's going to be and then you won't have to change it if you pass that additional parameter but in this case because I'm using change me of course I would want it um I would want to pass that or I would want to change that upon my initial login and then the registry dot json is that file that has my login information and then I'm also going to say to use a separate cluster network so while this is going off and doing its thing it should finish up by the time that we finish the presentation here but I'm going to go back into presentation mode so all this information is here and then also in the slide deck which anybody wants it um in the appendix I have a lot of the different like the example files that I've used in this presentation as well so we talked a little bit about the pools when we when we get the cluster up and running I'll actually just I may have run something real quick that's going to go out and create the pools but there's not going to be any pools defined except there's one that SEPH creates for its own use but then we need to create some rgw pools the rgw pools and the SEPH pools it's recommended to pre-create the rgw pools because we want to use a lower number of what we call placement groups than usually what the default is and then if you if you want to create the pool or the rgw service going across multiple different zones you need to be able to create those those pools with those own names in them and then again we talked about the erasure code of pool for multi or for the data bucket and then non erasure code pools for multi-part uploads and then for SEPH I asked you those pools have to be created when you deploy the service so there's a couple different ways you can do it but anyway I'm going to show you creating the pools and then deploying the service um so the overstair integration um you know the latest is 17.1 you know like sweet you know Fred hats having our hat fast for it and we're deploying SEPH 6 um so when you're doing external like if it's not about creating the pools and so for the four services and I know I'm I'm going to be over time here a little bit um you know for centered over glance and and the center backups there's default names that are used and so if you create these pools as these names you don't have to use any overrides in the open stack deployment but you can name them anything you want and then in the in your environment files you can actually set what those are and then on the SEPH cluster we want to create a client open stack user and then if you're going to use the manila services for SEPH S you create that user as well um on the open stack side there's these environment files environment files here and then you can create the custom environment file to override these so again it's kind of like going back to here to say like what these these pool names are and what the what the user is if you don't use the default um and then once that's done then all you have to do is the open stack overflow deploy and it's going to go in third like with 17 you have to do the deploy SEPH step and then it's going to take this information and then it's going to go talk to that external cluster and do whatever it needs to do on the open stack side to be able to just set up that communication and set up the sender service and manila and whatever else if you're going to deploy in this um whoever comes to the open shift integration um you want to make sure that like you install the open shift data foundation operator and then there's just a few prerequisites that you want to make sure um is configured on the SEPH side and that is that the SEPH dashboard is configured because that it's going to integrate some of that into the open shift dashboard and that the pg autoscaler is enabled and um that the rbd pools exist um and again it's recommended that when you create your pools and you're in you're using SEPH for let's say an open stack cluster and another open shift cluster and maybe a third cluster that's open shift is that all the pools are are dedicated to those particular clusters um so once you if you just install the odf operator um it which can be different as you don't install the lso uh the local storage operator you just install odf and then you go to create the storage system and when you create the storage subsystem or the storage cluster there's going to be a script that you download from odf or actually from open shift and then you take that Python script and either like go over to the SEPH cluster yourself or you give it to your storage admin and they run that script and it's just going to go on collected from um from that SEPH cluster and it's going to create the user that it needs to be able to communicate um to with the pools and gives the right um the right permissions and then you're going to get an output you're just going to get a json file of um information and then that's going to be uploaded into the um into the GUI and then you click next and it goes out and does this magic and it deploys the um the cluster and sets up all the services so it's going to set up your irees and it's going to set up your uh your gateways it'll it'll still install nuba for the multi cloud gateway um and you're off to the races so with that let's see um how we do it so we actually have a cluster up and running and that's how long it took to solve the cluster so i'm over a little bit um so i'll take any questions but i'll like i'll be more than happy like afterwards maybe to show you the odf um if anybody's interested but any questions for performance impacts or a virtual machine for instance running on a ratio coding versus replication so for like for what for rvd um for something like that that we only support they only support um rough so you would you would have you would have a performance impact yeah i imagine it doesn't support you know roughly what's the difference in performance like for via i don't i wouldn't even venture to guess and a lot of times whenever like anytime there's questions that come up on performance they i mean they always say your performance will vary and you test it out like and like and like i get that even like coming from the customer side you can only ask that question every vendor is always going to tell you your performance may vary so it's always best to test it out so any performance improvements on concept six versus step five um you know what that's a great question and i do know that they did make a lot of performance um enhancements and um that one document that we have to fill out um for you know what which one i'm talking about sure um i got a really good article from uh kyle wader and it talks about what improvements they've made in staff six especially for large-scale deployments um because of like some testing that they did with like thousands of osks and like when they go through upgrades of like how things are impacted on really large-scale cluster any other questions