 All right, thanks everybody for joining. I guess we'll go get started. I'm Jeff Borneman, and I'm from Red Hat. We work on, this is Michael Surby. We both work on OpenShift, and today we're going to talk a little bit about some of the exciting features we see in Staple Sets, how they apply to Timongo. So a little bit about the agenda today. So I'm going to go ahead and define what Staple Sets are. Then go ahead and give you some examples of use cases for Staple Sets. And then after that, take some time and do some live demoing. We'll also talk a little bit about some things I really like that have come out in the latest version of Kubernetes. So I'll talk about that at the end. Just a little bit of ground rules. I mean, we're a little informal here, so there will be some questions at the end. But if anybody has anything pressing, please raise your hand and we'll get to you. If this devolves into some sort of discussion, that's all right with us. Hopefully, that's all right with you. Otherwise, this is the last session of the day, so I think everybody's loosey-goosey and been learning a lot and ready to get home. All right, let's get into it. So what are Staple Sets? Staple Sets are a way for you to bring Staple applications or workloads to Kubernetes. They were originally brought to Kubernetes in 1.5, and then they're going GA and 1.9, so. Just a few nice features. Essentially, all around this is centered around pods having greater sense of identity. So how we usually think about pods in terms of deployments is that there are these stateless workloads. They can be killed when they're not needed, scaled up when they are, and so on and so forth. So thinking about pods in terms of identity, what do we have? Well, now we have stable replica storage. So when Staple Sets scale up, we can have stable storage that also scales with those replicas. Each replica also has its own network identity now, so instead of just being limited to pods being load balanced in a service, we have pods with their own addressable space. And then we have some sense of ordered startup for Staple Sets, and that sends to mesh really well with the kinds of applications that are most applicable to Staple Sets. So what kind of problems do they solve? Mike and I both work a lot with Red Hat customers and their journey through containers, and one of the things that we see that is most applicable to Staple Sets are vendor applications. So there is sort of a timeline of, or a progression or evolution of containers, and one thing that we see is that vendors do see the value in containers, and they do see the value in container orchestration, but there's some missing pieces between that and the support for the underlying platform that the containers run on. So this is a great way for our customers and adopters to lift, what we call lift and shift, their container workloads over to Kubernetes. We also, some other things to be a little bit more specific, storage with scale, is what I mean by this, applications that scale, they tend to need their own storage, would that be an indices or some sort of runtime state information? And then chatty clusters. So clusters, when they scale, they like to talk to one another, gather like graph information, and then order cluster entries. So entries into the cluster like to have some notion of the replicas before them being ready and started. And to add to your points, also everything that comes with Kubernetes like velocity, so the time to develop a product or a vendor product and get it out the door is additionally something that comes into mind as well as scalability from not only the software perspective, but also teams perspective. And then, yeah, just overall efficiency and automation kind of brought into play. Yeah, certainly. So like an example of a concrete example here, Cassandra. So if you think about Cassandra, Cassandra cluster, we have nodes in a graph and they use this protocol, a Gossip protocol to share information between the nodes in the graph and the edges in the graph. So thinking about how do we scale Cassandra in this method, staple sets add some value here. Another example is Confluence Data Center. So this is an interesting example because it actually gets all three points here. We have Confluence in a data center. There's a very traditional sysadmini sort of startup procedure. Every node in the cluster has to be started ready before a new node is added. Every node also needs to add its own stable storage. So it stores index information, losing index information per node. There's also some shared storage for a configuration that you'd see exposed to a PVCR or the like. And then of course they need replication. So they each need some notion of a network address and being able to communicate with one another. And that's really the question is how do we take these patterns and scale them? Because it's not really an issue of putting them on Kubernetes. We can certainly do that. We could manually create services that expose the replicas and all that, but maybe orchestrated through Helm, but Kubernetes or, excuse me, staple sets really add an easy way for this to happen in an automated fashion. So why would you put a cluster within a cluster? Why not? Well, in all honesty, it's the ability to provide a migration path for these clustered applications so that they could then take advantage of the framework or embedded features that are already out of the box with inside of Kubernetes and the vibrant community around them. In an ideal world, they'd sit just in Kubernetes and there'd be this lack of cluster inception, but it is a gap that needs to be bridged. Maybe we'll bring the mic over to you because it's some interesting discussion. And we want the webcast to grab it. It's totally a great point, so thank you for sharing. In the IoT world, you're bringing up data from sensors and you've got collectors coming up and one of the things that you may be running is something that is monitoring the sensor health information and you want to generate an alarm state. Well, those alarms are stateful. When I send it out to whatever system is coming out to warn somebody that something's bad, I have to know that they've received it. If they don't receive it, have to resend it to them. At some point, they have an acknowledgement coming in, so there's a fair amount of stateful information that is coming up from collectors in the IoT space. Because of the amount of data that may be floating up into these collectors and alarm managers, I do need to have more than one of these guys running here. No. Thank you. Anybody else have any examples? We'll get back to it. The questions, if anybody else has any ideas, again, this is a forum, so. Our example is going to be Mongo, so this is actually how Mike and I met. We did some work with the Mongo image here at Red Hat to make it a little bit more capable in terms of sharding and scaling with OpenShift. So just a quick debrief. You, of course, still need to be a Mongo expert to attend this, so. This is just the high level of what we'll see. So we have clients, and then we have a Mongo S component, and this is the shard server of Mongo. If you think about the shards here, you can think of them as buckets, where data comes in and then it's hashed and it's determined which shard that data should fall in. And then there can some config servers over on the right. This is where this sort of partitioning information is kept, and then we have shards, and these will be exposed as staple sets. So we can add more secondaries as needed without maintenance overhead, so. From that, we'll do a bit of a live demo here. So we'll see how this goes. The screen cutoff. Resolution's got standby displays. All right, so before this talk, I already set up a basic cluster. What we have, as I mentioned, is a shard server. Wait, before you go, can everybody in the back see that? Is it quite visible, or? So they need to do a little bit of zoom in and forth. So before you give me a thumbs up, I see a thumbs up. Thanks. Optimal, suboptimal, suboptimal. Does that mean zoom or not zoom? Yeah, zoom one more. Zoom one more. Great. Good? All right. Awesome. So yeah, we have a shard server here. This is nothing special. This is deployed through a plain Jane deployment. It's a stateless component. And then we have config server, which is also exposed. It's basically just a wrapped replica set. This is exposed as a staple set. And then we have one shard. So this isn't currently very useful, but we'll go through some of the features and see how they may interact. The neat thing here is to show off network, one of the things we can do here is we can actually kill the primary, have a little bit of fun, and see a re-election happen. So let's go, currently the way it works is when the storage is uninitialized, index zero actually starts up as primary. Maybe I'll zoom that as well. Shit. Okay. Good. So we'll RLSH into the index one secondary. So let's do OC get pods. Then let's get into ABC one. Actually, we don't even need to do this. We can just do OC logs. So how about we do OC logs, pod, and then tail the logs here. We should see some re-election activity happen once we kill the primary. So just to confirm that index zero is actually primary, let's RLSH into that. There we go. Thanks for nothing, team looks. All right. Excuse me. Right there. So Jeff is on the primary of the entire MongoDB replica set. So actually it looks like index three is primary. So that's the one that we're gonna kill off. And then we're gonna look at index one, and we're gonna see some traffic here. Correlates with re-election. So let's go ahead and kill that pod. Do you wanna do a OC get pods and dash wait and kind of see as the replica set or the Kubernetes replica set or stable set re-instantiates itself. So just dash W. That was fast. Still terminating. So as you can see, it goes in an orderly manner and it's not all at once. And you can tell what order it's in by the ending appended to the name of the, for the namespace. So we're seeing the entire. Otherwise it'd be a, in a typical Kubernetes replica set, it'd just be a random unique ID, so. So it looks like there was some activity here. Let's go back in and let's see re-election happen. Three is primary. Oh well, we'll try it folks. How about a ping? Can't go right. How about a ping? Can't go wrong with a ping. So we can, we can establish connectivity from another replica or another replica there. The other thing is that when stable set scale up storage also scales up. So we have, we have corresponding storage here. So this is not very useful. So let's add another, let's add another replica set. And then we'll watch some of these other features like or start up. So you can talk a little bit about. So some of this, yeah, so this is a, I know some of, this might be new to some people. This is open to a specific, this is a template. You can think of it a little bit like Helm, where we're gathering some parameters and then we're gonna use that to drive the creation of various resources. So we can look at those template afterwards to see what's being created here if you'd like. And then also the new replica set needs to, or can you explain the replica set seed as well? So yeah, since we are using a template, we need the resources to be unique from the template. So we have this seed that's appended to the resource name and that just allows for a unique resource to be generated each run. It is generated, but I'm going to use DEF because that's what I remember. It's going to one, it's going to four. Interesting way to see this is from the staple set view. The light blue is not ready. The gray is, the pod is starting up, or the container is creating. So we're going to two, and then we can also see the storage being created alongside of this. So currently we have zero through two being created. And then we have three. Mike's going to talk a little bit about what we just saw at more of a high level graph view. So essentially what we did is we took advantage of what the staple sets does, and that's from the Kubernetes perspective. We just defined that we were going to deploy the staple set and how many pods of the image that we were going to deploy. And that's listed in the blue box. And so those got instantiated with unique orderly numbers. So based on the namespace. And then in addition to that, we created a headless service. So we don't need DNS to be taken care of by, or a clustered IP, a virtual clustered IP address to be taken care of by Kubernetes. We are doing it from the application level. So we just go ahead and we ping that service and then it'll list all the members with inside of our staple set. So in this case, we don't need it to be load balanced. We're going to use just that headless service as a base entry. So it's kind of color coded. This might be an interesting graph to look at later. So blue, the blue part comes from the headless service and then the orange piece is generated from the staple set's name definition. And then three is the ordinal index from Kubernetes. And then also with regard to storage, it gets provisioned before the pod itself. So it needs to be available. It needs to be available before it's bound. And then when you remove members, the storage still exists. So storage is treated as a first class citizen with inside of Kubernetes, so. And just to show what we already have seen. As things come up, the replicas can communicate with one another. They all have addressable space. So let's, we can talk a little bit about that. So it's zero to N. So one of the guarantees we have is that if a pod comes up, N plus one isn't, has a guarantee that pod N has been started and is ready. So this basically lets us treat, potentially treat lower indices as, you know, masters or various other applicable things. So given that, what are some interesting use cases that customers are using staple sets for, like, from your experience? So. Yeah, so we see confluence. We see Jira. We see databases. Any of that sort of master-slave relationship that tends to be highlighted. And now just some other things that aren't, that aren't represented in OpenShift, but I think it's really cool and Kubernetes is we have this idea of an update strategy. So currently, the update strategy is default on delete. So if you delete a staple set, pods are retained. But with a boot one seven, which isn't rolled in yet with the OpenShift that I'm showing, they have a rolling update concept. So you can actually have the pods shut down in the opposite direction. So end to one will shut down in a readiness fashion. In addition, you have partition. So in the instance where, you know, index zero is say for a web logic, you have an admin server or something, you could have that associated with index zero and then partition off one to end to be applicable to your update so that you don't touch your admin server, but when you roll out, you have one to end. And then pod management. So this is, this really allows you to forego the readiness entirely in my opinion. So currently it's no ordered and ready. So we have that one to end relationship of waiting for the readiness probe to succeed. But if you just want to use staple set, say for storage provisioning capacity, then you know, you can utilize parallel to just forego that entirely. So those are some of the cool things that we see. And I'm really excited to see how it develops in the future. So we got a question. So let you go ahead and repeat it for guys online. Is there any like capability to have like kind of individual instance of a staple set, like addressing an individual instance and saying like I want to stop this particular instance. I want to stop this particular instance. I want to bounce this particular instance. Like when you're running like crap legacy software inside something, at some point here, someone's like I just need you to bounce this thing and this one node is having a problem. We played around with this with like labels and affinity and it would just like, you could trick it into doing that, but like having a way to say like, you know what, index zero, just stop it and let it restart. So maybe go to the OpenShift console and then click on the representing staple set, pod on the right hand side. So you can just kind of delete. Bouncing would just be deleting, right? So. And then communities would take over and re-instantiate and then the logic that's built into the instantiation of that member will determine if it's either a primary. This is, yeah, specific to Mongo, but in general. That's just, well, you could do it with a cooped CTO and then kill the pod, yeah, just delete the pod. No worries. It's still like a replication controller in that it's guaranteeing that a certain number of replicas is up. So there's just that additional layer of making sure ordering is still preserved. If you have that set, that would be a scale down. So if you have like special snowflakes within your staple set, then if you're so inclined, then you put those in lower indices. And then if. Go ahead. Oh yeah, good boy. One second, my partner's gonna come to you. So when you delete this pod in this case, the underlying PV gets removed or it will be retouched to the new pod that will be reinstantiated? It actually gets preserved. So it's sticky to the index. And that's a really important sort of CYA sort of thing. Deleting any of the pods of the staple sets underlying provisioning of storage is capped. So if you have a database or something distributed like that, then life is still good all the way in the back. Could you tell me what the problems might be if you try to deploy a stateful application like MongoDB or DB2, but not using a staple set? Which is something we're doing right now. We have the MQ series, DB2. We use deployment, which I assume is for, and again, I'm somewhat new to Kubernetes which is stateless, right? And it seems to be working fine, but I heard for stateful applications you better use stateful objects. So I can answer that. Essentially you get a lot of refactoring of the stateful application or staple workload that you're working with in order to suit Kubernetes replica sets. So this was driven by demand from the community to be able to basically provide you an easier migration path to take advantage of microservices architecture associated with one of a stateful application. So basically this gets you to get onto Kubernetes to begin with and then just have a more phased or a methodical approach to decoupling those tightly integrated services, so. Yeah, just to add to that a little bit, and when Mike and I were originally working on this, one of the biggest things was provisioning enough storage. So previously we had customers where, no kidding, it would be weeks to months of a wait time on storage team or for storage or even really for a team for compute to add a new replica set. With this, deploying those types of applications and having a click and then having a new replica set is huge and it's really exciting. And also because it's using Kubernetes you also get a cloud native application like from just putting a stateful application inside of Kubernetes to begin with. So now you can take advantage of provisioning across those cloud providers as well. So, let's have a go over there. Yeah. Say again. So I would just add to that that there's a number of applications that don't change their DNS name well. They'll go through and create new UUIDs or show up as new cluster members or if you're going through and trying to configure and say, hey, we want these notes to come up and find each other. The dynamic DNS portion that comes up with running a deployment, we're gonna get that last five digits that are different in pods can be different whereas the stateful set, you're always guaranteeing it's gonna have the same name. You can put that in a static config map or something else and make sure that you're always finding the same box. Yeah, so I'll have a question. Could you talk a bit about network configuration, particularly for chatty services like ZooKeeper or RedCD that need to notes each other? How the network configuration is managed on a stateful set? Well, you can dive into details, but if you wanna just get going and running, it's really just a matter of creating that service. And that hello service is used to create those DNS entries per replica. And that's done automatically. So there's really no getting your hands into the details as far as that's concerned. But if you're talking about more of a managing how the pods are scheduled so that there's low latency links, then that's not really a detail that you work with specifically with stateful sets. That's more something that you consider with labels and your scheduling policies. But it's definitely something to consider. Any other questions? Any other questions? Jinx, you owe me a soda, Jeff, later. A soda? Yeah, good. What types of volumes are we using? For this, this is a host volume. So this is many shifts, essentially the analogous to many cube on my machine. But there's a lot of initiatives being driven around cloud native storage in particular from the Red Hat end. And we also have supported products as well. We've done this on EPS volume, so. But basically. That works just like you usually would, creating the PVC. Those PVCs are just dynamically generated from a volume claim template that's defined in the staple set. Sorry. Is the instance deleted and then started on rolling updates? If they're gonna keep all their unique identifiers? Deleted and updated, yes. Well, there's also a history associated with deployments. So it gets saved if you go to the... So it would be like scale down. So if you have like one, two, replica three, then three would get fully shut down, deleted. Two would get fully shut down and deleted. One would get fully shut down and deleted, zero, et cetera. And then it would get recreated from the bottom to top again. Zero, one, two, three. Does that make sense? And all those network identifiers are sticky. So as they come back up, they're rebounded to that same address. Any other questions? And by the way, this demo can be done from this URL. All the instructions I have here. So if you wanna play around with it on your own. Of course, these slides are posted as well. Well, thank you so much. Yeah, thanks a lot. We appreciate it. Happy keep coming.