 Good morning. Good afternoon. Good evening and welcome to another episode of red hat enterprise Linux presents today We're gonna be talking about rail system roles and we have no guest today So it is just myself and the wonderful Scott McBride That's true doing good. You're you're burdened by only having me with you today Chris It's terrible. I know I I mean, hmm. I wish for so many things in my life, but yet here we are Well, we are going to be doing live demo later. So hopefully that just goes up in total flames and then Idea, right? Like we're trying to just get people comfortable troubleshooting stuff, right? So yeah live demos are great So system roles, right? Like Well, first is there anything follow-up wise is you want to touch from last episode? No, we talked Last episode was with John Spinks and we're talking about Management and insights and I think we will touch on it a little bit when we we start talking about system roles a bit because They use as well. We use it as a goal We do your ins all together Yeah, I think we're pretty good shape there. Yeah, cool So system roles are not system purpose Right. You distinction to make right and When we were initially doing our setup for the show We were talking about this a little bit and you may recall that Two or three episodes ago we talked with Rich Dorito and Rich Dorito was we were talking about subscription manager And then there's this Affiliated tool called system purpose or cis purpose, right? It's this purpose is used to identify This box is a production box or a test box or you know something along those lines So when it registers with Red Hat subscription manager It will look at the pool of available subscriptions in your account and choose one that it thinks is the best match so that you're not tying a premium 24 by 7 support subscription To your dev box, right? It doesn't need that. Yeah, and so that's that's what cis purposes is to make Better automated decisions on what subscriptions get pulled out of your account and assigned to end visual systems as they register system roles are using a technology included in the ansible suite called ansible roles nice and They basically are like extensions For playbooks, okay, so ansible has a whole bunch of different things that they use to extend Beyond just like their YAML and playbook infrastructure, right? They've got Modules they've got plugins and they also have roles and that's what system roles is. It's worth. It's insible role Cool so These are pre-baked ready to go for me Like how do I start consuming them? An excellent question. It's one that I think we don't do a great job of Answering in a lot of our materials like we tell people what they are like which ones are available We never tell them how to use that, right? And so we get to the demo actually be showing like how it all hooks up in their playbook But they are remade by Red Hat There's actually an upstream project for them and then what gets sent as part of Red Hat Enterprise Linux so As we're working on new ones, you'll see it will appear in the upstream project Prior to their appearance in Red Hat Enterprise Linux Typically when they appear in Red Hat Enterprise Linux, it's at a dot release of the product So 8.3 for example had a whole bunch of new ones that got added into rel that were pulled in from the upstream If you look at the upstream today There's a lot more kind of in-flight there than what is available in rel today Right, and we'll see what what lands at different dot releases going forward but for example, I Know that they're working on one for the rel.ha plugin or rel.ha add-on So you can do some configuration of your pacemaker clusters And if you look at the upstream, there's not a lot there yet because it's still very much in development so I would not expect that one to land and in a recent release of rel right it might be a Couple releases down the right road and and I just found that they have a web page that kind of breaks down everything that's in the repo for you Including like here's demos. Here's currently supported ones. Here's ones that are on the roadmap So be sure to check that out if you're interested, right? is So these are answerable roles Like I get them from Red Hat I apply them on my system using Ansible not rpm or yam or anything like that. So Do I necessarily need to buy answerable support or are these Supported because they come with rel kind of deal. Okay, so so that can be a complex question So let's let's go with a simple answer first and then we'll see how how far down the rabbit hole we go So you don't need to buy Anything beyond a rel subscription to use and consume rel system roles Essentially your rel subscription will also provide you access to an ansible repo For the purpose of running system roles so that you get a supported ansible to running a supported feature of growl Which is system roles? If you're doing stuff outside of that You may need an extra animal subscription But for rel system roles what comes with rel is what you can use And then to actually get it going We'll have to install the rel system roles rpm which gives you the ansible role content And then you also have to install ansible so that you can run playbooks Okay, cool that makes sense So one step to get the thing another step to actually execute the thing Yeah, and when we hit demo time actually start from like here's how we subscribe to the ansible repo, okay, now let's install the pieces we need so It's not too bad. It's like to our PMC need Yeah, so If folks are looking for the official docs As far as like real stuff goes since we've talked about upstream. What's I'll drop those in chat What in your opinion are like, I mean, this is your opinion literally I'm asking you now What's your favorite system role? I mean there has to be one where you're like Most people who needs this It's good question so I like the The session recording system rule it uses t-long as the back end and does the configuration for it, okay But it only applies to rel 8 Let's see. I like I Like the kernel setting system rule So it actually uses tune D behind the scenes to apply changes the system But the reason I really like it is the automation piece the system rules gets right so I worked a lot of places where you would have a farm of boxes that are databases and maybe You know, let's say My sequel Somebody finds a document that says my sequel should have the swapping this value of the kernel set to 20 Right, so you go around and as you're building all those boxes That's what you do and then you find another article From a more reputable source or more recently that says no, that should really be 10 Not 20. Okay. Well in the olden days you then ran around all the boxes and like did the needful on them And now if you had implemented a system rule for that You can literally just like update the playbook execute the playbook across population so That's why I like it and system rules has kind of an interesting history of how it came about Yeah, let's talk about that. So Originally it started because As we released new major releases of brawl, right so from six to seven There was a lot of change that happened there. Oh, yeah. No, I remember that Well beyond system D like right, you know, we added in crony and yeah, you know We make different things the default things even though we still ship older things So crony is a good example where we have NTPD and crony and you get one of the other And then, you know, when you go to rail eight while I think the differential between seven and eight was not as large You still get some of those technological changes absolutely and so The original intention behind system rules was let's provide a method That administrators could use to do something like set their NTP servers Like right, that's a pretty common thing to have across your population what what NTP server should they use? and so in six seven and eight Those were potentially different methods of applying that change different technologies different files and you had to know Yeah, I remember having rail put out a nice like six seven eight like comparison chart between commands I remember having to pull that out during those upgrade processes many times And we still have that it's and it's great. It's a great resource. Let me go find it before I forget about But like from a customer perspective or from a you know administrator user perspective That's super annoying because here you are you've got this large fleet of systems on rail six and then real seven comes out and now you have to know oh There's a real six bucks. I need to do it this way. Oh, it's real seven bucks I need to do it this way and when you're trying to use shell scripting whatever else to automate Your shell script needs to account for that in its automation Yeah, right. Otherwise you'd get the long the long stuff done on the long box so One of the things with system rules is that when you use it It is cross rail capable So for example date time sake system rules a good example of that You can just put in the NTP servers you want your boxes to get and if it's on a rail six It does NTP D configuration stuff if it's on rail seven It actually checks to see whether you're using NTP D or crony and does the right thing and on rail eight It uses crony So you can use the same playbook across those different generations of red-hot enterprise Linux and Get a expected working output without having to worry about what's happening underneath Same thing with like kernel settings. It works on seven and eight Same deal and there is some What roles are cross portable and which ones aren't is gonna depend somewhat on when the role was added to the catalog. So for example Kernel settings was added in Rail eight dot three and I believe it had also been added in seven time nine But Well sex was already in maintenance phase two, which means no new features Including system roles. So it was not put into well sex Well seven has now crossed into maintenance phase two. So we'll know what they're beginning new features So any new system roles we make Or it's gonna be back toward in real seven and then there are some like session recording where That that technology doesn't come with anything but rel eight at this point So if you run it on a real seven or six box, it's not gonna do anything. I Cannot in the access that I cannot for life me find that PDF. I don't know if I'm searching wrong or not If you have it handy, I would appreciate I can probably Google food out for you Well, I Google food it and I found five six seven. I did not find six seven eight So yeah And I remember the five six seven chi chi well Don't recall there being one for so seven eight six seven eight. Sorry Four five six seven eight. So yes, I see the five six seven one. Yeah Maybe we didn't I'm sorry for the dog. Ah, here we go. Oh, you found it good. Maybe let me check. All right Yes We We can just put it in here, but quick start so That's what happens when you have more than one box. I know right. Oh, there it is. Okay, cool Also, yes, this is exactly what I was looking for cool Yeah, I still use that five six seven web-based chi chi Occasionally, but mostly because it's like wait five was 12 years ago. What do we do then? Well, I'm thinking of the the folks that are like changing jobs and they walk into Upgrade, you know a fleet of servers and it's like, okay. I was using seven and they're on Five still and I'm trying to get them upgraded kind of thing right like I've been in situations like that So having this chi chi really helps you like If you've been living in the future or the past this can bring you up to speed kind of thing I uh Like indulges to anyone who is working in a real project right now. Yeah. Yeah. Yeah, I mean, I remember what was it? 2011 we were upgrading from rail three two five and just Skipped over for or whatever and was like, okay That that was fun But then it became easier to upgrade because real five made it easier real six made it easier Upgrades got simpler. We got more automation so forth so on yeah and I actually was just asked yesterday about Six to seven upgrades and it's like Okay, so six is it extended life support phase So very limited updates essentially critical and an important security or out of all late With an add-on subscription, right? So you don't have to add on you don't get any updates anymore for it rail seven is in maintenance phase two. So it's got I don't know two two and a half years left before it goes extended life support and end of life phase so If you're going from six to seven You're essentially by yourself two years two and a half years And they're going to be the exact same situation again. It's like If you're going to go through all that effort Maybe it's worthwhile to go through all that effort to get up to eight which is currently in full support phase for another See year and a half Uh, I forget but I did stumble across the life cycle page when I was searching. So let me go Yeah, I was released in 2019. So 2024 is when it goes out of full support and goes into maintenance support for another five years So, I mean, you still got a lot of life left on rail eight and We're doing work to make the transition between major versions easier I don't know if we've talked about it before I know in March, I think it is we're going to have terry bowling on he's the And we're going to talk about leap which is the in-place upgrade utility. Yeah So we're currently using that for seven to eight. We're going to continue to use it for eight to nine But because the differential between eight and nine is not as great. It should be easier and smoother for people to use for eight to nine Yeah, and it's like we've gotten way better point releases too, right the whole release process, right? It I feel the evolution right like happening underneath me as a user. All right. It's pretty great Yeah, so Is there any other questions I needed to ask you specific to system roles? Let me double check here We kind of touched on what needed to be installed. You needed the system roll package as well as ansible. Yep I'm supporting well subscription Right. This is part of rail. You don't have to buy ansible The purpose of system roles we kind of discussed Oh, we we got sidetracked on that one. Yeah, we got started as that like let me run some let me have a slightly different administrative interface that I don't have to care about version Uh, so that as we had these major versions, uh changes You don't have to go through and update all of your scripting procedures automation Right, it all just still kind of works Um, right, but out of that has generated kind of a new idea of what we could use system roles for Which is standard operating environment. So if you're going to set Kernel settings across A segment of your population of your entire population Like let's build it into a playbook that we apply to all our boxes Or if you're going to set a specific configuration for ntp servers, you only want to use the pool servers or you only want to use the public one in your higher priority, right? Yeah, or you want to use your sp1? Will whoever You can you can do that and you could put it in your Your ansible playbook using system role to apply it across There are also some newer system roles that Do administrative tasks? So like the storage system role, for example, you can create Partitions on disks you can add new logical volumes you extend logical volumes. So But I will caution you that it is more utilitarian more It will provide you more benefit Wow, just like not using my words today. Okay, porky pig Uh, it will something like storage will provide you more value the more Complicated the more homogeneous. Yeah, yeah, your hardware is so if all your hardware has the exact same disk layout has the exact same Number of disks the exact same devices of disk Then the storage role is really really handy as you can execute that change and kind of know That across your population That new partition is one going to be able to be made and two isn't going to pop or something else or do something wonky Yeah, so if you have a really disparate population of all random stuff Then that probably would not be the best choice to to do Yeah, we had My first job out of the air force we had different like classes of servers and that's where I see that being helpful, right? Like if you have 20 Batch processing boxes and they need this amount of storage and this much in temp and this much and you know var That is super helpful, but that Could be something completely different in your database fleet or your you know, web server fleet or whatever you're using, you know That just and maybe you have a Standard operating environment that gets applied to a specific type of box. You have your web server S&E you have your database s&e but um, I worked for a large media conglomerate and their process for acquiring new hardware was buying Whatever was left over at the hardware manufacturer so like Yeah, so we would get things like dl280s HP dl280s, but they would have Different number of next different amount of ram different number of cores different disks and so You know While we could guarantee some basic sameness like they're all going to be the same processor generation Or there was going to be a minimum amount of ram that was in there We wouldn't necessarily know the the complete package before we got hands on it and to put it out the data center And something's like nix. You can't Oh, you can never guarantee. Yeah Right, like I'll know it has at least two because that was in my spec when I when I put it out to be fulfilled But it might have sex So, yeah, anyway, but fun times about standardization From the guy who used to work at nest nice Okay, cool, so are you you want to dive into this demo or sure because I feel like I'm gonna ask many questions during your demo Oh Absolutely, so let me just uh re-privilege this box here and we share my screen screen sharing Yeah, okay. So you should see it like currently provisioning Yeah, sweet hang tight Yep These boxes only exist for 30 minutes at a time So, you know, we're talking for 20 minutes. It would come suck in 10 minutes in the demo just then Yeah All right. So this is a a lab experience that I have not completed if you ever used a lab lab at redhat.com before typically there's like a lot of Crows in the instructions that tell you like why you're doing this step and what to look for in the output I haven't gotten to that yet. So at this point this this lab is essentially like click to run this command That might be super complicated and has no explanation of what it's doing But here's the output And there's no explanation of the output either. So we'll do that live on on the demo here. Cool The other thing I'd like to do with this demo is it's on a multi-node environment So I'd really like to get it Once I've gotten all the instructional stuff ironed out I'd like to get it so that we can apply the same playbook across all those So that's that's my plan. I had originally intended to have it completed by the end of 2020 You can see how that worked out for me. Well anything With the word 2020 in it as a goal You should have tossed that out in march, right? Fair enough all right So I said that the first thing we need to do was make sure we have the right software And part of that is getting access to the ansible rpm that we need that provides the ansible executable So I'm going to use the subscription manager command to just enable the ansible 2.9 repo for this Roblox It's included as part of your real subscription Or you have to do is enable it and then you're given access to the software that's contained with it very nice Yeah, you can do a subscription manager. I believe I blame rich giurino That's a good one All right, the next thing is installing the software you need So I'm going to install both the ansible rpm and the rel system rules rpm And they'll pull in a couple of dependencies as well, of course, yeah Python 3 that's a good one to pull in. Yeah They needed that for the ansibles So before we go further, I do just want to point out like where things are going So if we take a look at the list of files that were provided by rel system rules There's some documentation that's installed that's text-based. It's pretty basic But here in user share ansible roles That's where the actual ansible stuff that makes it work is and so Earlier we were talking about the differential between upstream and what comes as part of rel There it is So these are the ones that come with rel. Some of them will be tech preview So for example the post-fix one, I know is tech preview The other ones I think are all all now fully supported But that would be referenced in a rel documentation that you learned earlier chris So if you're interested in which ones were tech preview or not, you can find them there And so inside these directories is all the the ansibleness that makes the roles work with ansible Okay, so this is one of my invalid things. It's not working right Awesome Yep always So I'm just going to copy and paste this content Fair enough There we go All right, so this is a starting point that I wanted to have in the lab Especially because people may not be familiar with ansible and how they would write a playbook So I wanted to at least give them a starting point to kind of work off of And what we can see is down here In the roles section These are the role system roles that I'm pulling into my playbook. Okay So it's going to go out into So terminal logging and kernels Right, right and um because I have imported these roles that means that I can do these sections So I can now reference the kernel settings sys control and t-log scope sssd sections And then underneath of them I can define more variables And you find out what those variables are In the documentation It's crazy So kernel settings allows me to change Stuff that 2D can change Yeah, 2D is a great piece of software, right like Agree, yeah If you don't have 2D in your environment, I would encourage you to look into it And I was just talking with Mike Jaret over in the ISV team today about doing a webinar around developing 2D profiles specifically for People who are writing and distributing software at realm because I think that Including a 2D profile for your thing Has like that like extra level of polish and it's really not hard to do right like I can totally see a database ISV Being like oh, here's our 2D profile and everything else that we need right To make your thing perform You know at a certain you know performance level Or maybe there'd be multiple profiles so you could be like oh, is this a high right? Is this just a read-only replica like what is this? You know that kind of thing Yeah Or they can at least give you a basic one Yeah, and then if you need to customize it based off your use case they can provide that in their documentation or something else Yeah, like all of that sounds great to me Bake that in folks. There we go. Yeah Um, so because I use this kernel setting system rule and I'm now able to do these variables Um, I'm just going to set some parameters in the proxess directory Right, and that's what is happening here So I'm going to change the swappiness value for this system um swappiness is a value between zero and 100 that determines the kernel's affinity for utilizing swap space So zero it's going to try really really hard not to use swap space ever Ever like right as as much as ever can be right until a killer shows up Yeah 100 it's going to like try and use swapping this is a much spot or swap space is as much as possible right and so for For databases typically you want this to be 20 or less Depending on the database database depending on the workload. You might be choosing between 10 and 20 ish um Along with that I put in some other tunables for databases here. So In order to kind of ring as much performance as you can out of databases A lot of the open source and enterprise databases also tell you to make some changes to how your Cache is written and managed your file cache is written and managed And so the vm dirty ratio is don't sync file cache until it's 40 percent dirty And that are 40 percent of memory Is being used the expire sent to sex and write back sent to sex So when we decide that we need to do a a sync of file cache How long should it be until we decide that we need to do it? And so here we're saying that 30 pages expire in 500 sent a seconds And then Trying to remember what write back is it's another setting around kernel kernel pages. It escapes me at the moment And then the last one is the setting for semaphores so semaphores are Type of shared memory where processes can pass information back and forth to each other And so we're setting things like the size the total number of minimum and maximum length of semaphores and it's a Set of four values to get stuck in this one kernel setting nice so um So yeah, once I applied this playbook those are the settings that are going to be applied to this system through system rules The other one uh is my always be talked about as one of my favorites Um, this is session recording. It's based on a technology called t-log And essentially with this variable. I'm saying Start doing session recording for everyone so Why would you want to do that just out of curiosity? Okay, so uh same same media conglomerate whereas Serving a professional services engagement um We had a team of administrators that worked there And it was often the case that you would get a call like two or three o'clock in the morning Because it was not two or three o'clock in the morning somewhere else in the world And they need you to do something Um, and so here you are bleary eyed Getting out of bed executing a change on the systems. And what if it goes? haywire So then maybe you need somebody with more experience So you call an escalation And that person receiving your escalation asks you a question like What did you do? You could just say here you go. This is what I did Right and so they can actually replay your session and see not only what you did in terms of what commands you ran But also the outputs and errors that came from that Yeah, I was about to say the commands were only so good that you need the full output Yeah, right and and that's like Back then we essentially would like pull up somebody's history And try to detangle things based off of their history that they used But you're right without the output and error messages It gets really hard to figure out what exactly happened to that box But looking at at these recorded session you get everything in context Nice And so not only did that help us or would it help you? Detangle the situations and maybe repair them But you could also then use it as a training aid to be like look here you ran this command and you received this error Right and so this is what this error is telling you Um, and yeah, you can also here's the next step to fix that and then yeah, like that's really cool You could totally put together a program now Exactly around logging now and it all happens through Through text logs By default it goes to the same logging provider is configured on the system So for our boxes by default that'd be ursis log Which means you can also collect those logs not just from the individual boxes, but To a centralized logging host So the other thing that is really handy for this is like maybe you're in a more secure environment And you need to make sure that things don't go bad And if you allow someone to have root on an individual box, they could manipulate what's stored in their log After it happens But if you simulcast it to the local log and a remote log server You always have an unmangled copy on the log server Compared with what's locally on the system So and this would be for every single user right so like let's say somebody's credentials got Correct or you know, somebody came inside the wire You could then have an analysis on what's coming out of your logging service your centralized logging service if you have one And say hey, this isn't cool. Somebody just ran this command and you know Who knows what happened? So that's that's that's very handy. I feel like in larger environments, right? Like that could get To become a very useful thing Yeah, and I remember having to script these things out for myself sometimes and like Just having that output like in line as opposed to here's the errors You might encounter as like a separate section documentation, right? Like you just roll through them all that'd be cool well, and You know, we've had this in previous versions of row Kind of you could do a whole bunch of gymnastics with the audit d demon and get a lot of the same data But it would also do things like capture passwords, which you probably don't want Right and yeah, it goes into the audit logs, which are different in the system logs But still like you probably don't want that stuff um, whereas T log and session recording are set up to like Do do this in a more sane way than just capture system events and report on it Um, but this is something that's in row eight. It's not in previous versions of row There is actually a lab for it. So if you're really interested in it, you could go through the lab yourself on creating it and the back there's a command line playback utility called t log play Or there's a really really sweet plugin for cockpit or web console where you can get in a video. You can do zoom It'll you can actually put in a search term and because this is all stored in text It'll highlight the time codes where that search term exists. Like it's is really slick So anyway, that's awesome. Yeah, cool All right, so pull in the train back on the tracks for system roles. Yes. Thank you All right I see why it's your favorite though. That's I mean, this is good to know. Yeah. Yeah, it's got a lot of uh Got a lot of the use All right, so I've got my initial one and just to kind of see where we're at If I look at the swapping this value on this box right now, it's set to 30 After I run my playbook it should be set to 20 all right, so I'm gonna just uh As well playbook s we yaml right to execute my playbook um, and you can see that it's going through and Running different parameters Yep from kernel settings and then it's going to move to t-log um And then it should be completed at that point All right with uh with t-log I didn't have the software installed for it. So up in here where it talks about package facts and then installing the session recording plugin like That that's actually installing on the software you need to make that that happen that's built into the system for um Very cool first session recording. All right, so as expected when I look at the swap is by nano set to 20 And let me just real quick reset my password for this user All right, uh, and when I connect to the local system Right session recording was installed. It's there you go users. It's reporting that my session is being recorded um Session recording cannot go back in time is what I'm hearing right What do you mean like this is something you have to install before or correct You get the benefits of it. It's for any new session after it's been installed and configured that it'll uh That it'll pull it into your logs, right and just to to round out our discussion there. Um What is a t-log? um It's cure I don't think it is It's not I don't know what if we have time at the end. I'll Pull that one up and we can do that one too. All right. Uh, all right, so That that's like we created a playbook We set our parameters We told it which roles to use so we can knew what parameters to use And then we executed it and the parameters were applied to the system. All right So the next thing I'm going to do is I'm going to take that same Um yaml file and I'm just going to make some changes to it The first thing I'm going to do is I'm going to add another role to it So now we're going to do things in the playbook for kernel settings for t-log and for time sake nice And then I'm going to this is a super complex uh command But essentially what it's doing is it's uh A clickable so that I can shove stuff into the playbook without making type it and this type it. Yeah All right, so not as easy as you think right, so So when I uh executed it right there is my addition of the new role that I want to use time sync and then that said inline edit added this section Got it nice Where currently on this box is probably set up to use a bunch of ntp pool servers But maybe I have very specific ones I want to use so I'm going to use time.ness.gov box And then one out of the pool as like a backup right in reality probably went three maybe four or five Yeah Kind of depends on your need and sensitivities and such to time right Uh, so if we look at our crony sources right now Right, these are all pulled out of the pool and one of their yeah three six eight of them Um, but because they're pulled out of the pool Um, and I noticed they're like stratum two three I've seen right you have no idea six Right, um, whereas I know time time d Be.ness.gov is a stratum. Why I did just like connected the clock All right, so we can do a whole show on time and let's not slash ntpd and so forth, but yeah We shouldn't but let's not because it's super boring Oh, come on. I love time. That's fun I So to be honest I was the guy that said hey, we probably should get a gps time receiver since we're a news company at first job Out of military kind of deal, you know, because there's all sorts of system timing issues and I was like Hello, have you all heard of ntp? It's like, oh, yeah, we should do that. It's like, well, we should get our own time source too that we can trust And then have backups, right? So yeah, we actually well like the The ntpd servers oftentimes Random clients aren't permitted to connect to them, right? The only except connections from a list of of known servers So if you want to be one of those you need to like fill out an application and review it and decide where they want to do it So if you were say, um, I don't know A large vendor of some sort that wanted to maybe add all of your systems tens of thousands of them Well, and yeah, you know, right? Yeah, yeah, but if you're like a a large news organization where Time is important, right when you're like time stamping the publication date on articles or on news stories or whatever Um, you could register maybe three boxes Has an organization to that and then inside you use those three boxes. Anyway All right, enough on time for short. Super boring. Stop it All right, so I'm just gonna apply my updated playbook and It's basically going through and running all the same things and it should be reporting mostly Okays and skips because it's not doing anything except the time stuff Yeah, so, um, especially when you get down to t-log, right? It's already installed. It's already configured. We didn't make any changes Should be good All right, so now we're installing crony and making changes based off crony All right now that this part of the output here That makes sense Right, it makes sense. But if you are a person who's used to looking at red and green and red is bad It's bad, okay Right, but in reality, this is okay because what it's done is it I said earlier that we Go across to the versions of rel some versions of rel have ntp some have crony some have both which we're using and so essentially in here we're We're looking to see what's configured and then based off of the configuration. We're making the changes But it's actually checking and failing because it's not there right, um, I know that there is a open feature request with system rules guys And gals to change it so that this is simply a skip or an ignoring without the red error messaging So this is actually a thing in ansible, right where you can accept a failure, right? Like a failure in this thing does not mean an entire failure of the whole process It could just be a failure to check that the thing is actually there So i'm not going to continue down that role Or down, you know some other role that it needs to pull in it'll In theory quote skip over it, but it's not skipped because you have to check something with ansible to actually skip it So therefore it's failing to see the presence of something So therefore it ignores it and continues on because it's not a failure That is detrimental to what you're trying to do Right, but if you're just scanning the output that would attract your attention. You might know it's expected Right, but yeah, like disable time master, right if If you don't know what time master is you'd probably be like, oh, what is that and then you get to learn about time Well time master yet another service that one could use to manage time. Yes, exactly. Wow, right All right, so at this point if I check my current sources again Right, there's there's my two that I said I wanted one is time.ness.gov and the other one is random out of the pool All right, just like my playbook step All right, so When you want to make changes, right? I said earlier, maybe you're set up your database servers So if they use 20 and then you read the sub article and it says you should use 10 So you're like, all right, cool. Boom. There we go. I'll make the change Right that little in-place edit Just updated that guy And then once you've made your change you apply the playbook again And just like before it skips all the stuff that it had already done in the still the same state And then applies the new changes or the the differentials. So now we shouldn't see Time servers being set or changed or anything like that because it's already been done It hasn't been updated in the playbook has not been mucked with correct right So by adding more roles into the role section, you all of a sudden get to add more content into the variable section To customize different settings on your box All right, so no no surprise there that Changes anything down. Yep. So that's that's basic systems So let me pull up that. Yeah, we'll follow one more time Yeah, yeah This is an answer we'll playbook and with variables and everything else to find. It's awesome. Yeah, right and so Uh The documentation that's provided tells you about The layout I find it easier to read from the upstream system rules project because it's like You know, I pull up the github. I click on it. It gives me the markdown based Description of it that's a little bit easier to read than a text file on a terminal Yeah, and that's The hierarchy and everything right is there for you as well Right and if you have the awesome extension for github called octocat It will show you on the side where this thing would live in your infrastructure. Which is super cool noise Because I use some really simple system rules here the ones like storage Or logging configuration are a lot more complex and have a lot more variables and nuance to them And so the documentation goes much more key when working with those guys Yeah, no the The amount of things that you can Check and do and the safeties that you can put in with ansible are just vast. So, yeah the um I would be interested to find and I'll probably look after the show the most complex system role right and just like Highlight that on twitter somewhere. So, yeah So I think a good place to look would be storage and logging. Yeah, if I just You know, we're going to throw a couple out there. Um, I mean kernel could get complex But that depends on your variables, right like well Kind of but not really because it's really using toondi And so toondi has are an already established syntax that it uses. So for example for for these In the documentation it just says use use the sys control to the ball Right and that's what I'm using and tell me what to set it to so if I wanted to change I don't know. Oh make this machine a router I mean to turn on ip 40 Shall we shall we want it? Let's see Go ahead. See what? All right. Let's see if I remembered it. Uh rocks this ipv4 ip forward All right, so it's currently turned off. Yep. How good is the typing? I knew you're super impressed that I remembered that path I am actually to be honest with you because I would have had to have like tabbed like six or seven times to figure it out Network network forwarding is not something that I always did When I was administering rail box Well, yeah, I mean unless it's routing between networks, there's really no need But for a number of years I worked as a There it is. Cool. So that's it. I worked as a red hat Instructor for training and certification And we needed to route between networks and after you did your setup you went in and you twittled this bit or if things were working properly You would check this this value um, yeah, so There you go That's awesome Like this is very like I really like system roles like at first that it wasn't really you know I was ambivalent to him right like I've I've always been the type that's like gone to ansible galaxy and grab my role or playbook and You know scoop it up from there and kind of tune it and tweak it now having these like Absolute rock solid roles I'm probably going to refer to these more often as good documentation, you know Yeah, and like we're continuing to grow the catalog of roles Um, so I know that we're in development on one like I said earlier for the ha add-on. There's one for firewall settings There is one already out there for se limits because like Let's make sure it's enabled. Mm-hmm Don't don't don't make dan wash cry folks. Yeah, exactly. Um So, yeah, there's there's a bunch out there kind of circling um We added we're starting to add some for specific things like you may have noticed the certificate role Um, that was added in with idm for eight either a two or a three um, and so, you know how You have to manage your ssl certs And if they expire it's like really bad. Yeah, right So that system role can manage certificates. They're issued either self from a CA a local CA or through the cert monitor service, which is part of idm Yeah Yeah, cool. I just I just pulled up the the kit up repo. This is like Everything you would possibly need. I wish I had this five six years ago Exactly Exactly one of the last projects I worked on before I moved up to up here to michigan. We had we actually had to like Standardize web server configuration for like hundreds of hosts and that meant ssl certificates, too And this would have been super helpful Right. Well, I'm getting things like a wild card certificate to supply stuff so much. Yeah Which back in the day we didn't have yeah, I mean we were we were Using a vendor at the time, but we were like Seriously considering switching to let's encrypt, which if you haven't by now heard of let's encrypt Please go look it up because it could save you a lot of time and money So cool. Anything else you want to talk about as far as uh system roles go No, I think we're good. Did we get any questions or we're good? No, there's a lot of you know affirmation that you know, this is helpful. Yay So you're doing well, but like affirmation Yeah, yeah, always a good thing Yeah, so Next show we're going to have terry bowling on He actually He's a product manager for rel He's moving between product experiences. So system roles actually were his And he's handing it over to a new product manager Cockpit and web console were his and that's going to go with the new product manager He's also the product manager for image builder that he's going to retain And he's picking up Leap for in-place upgrades and convert to row Or converting from other rpm based Linux's to row And so in two weeks, we're going to have one. I think we're talking convert to row And then he'll come on again in April maybe Um to talk leap because uh So the new rel cadence is minor release every six months Major release every three years So Rel 8 was released in 2019 in the spring Guess what's happening in the spring of 2022? Rel 9 so So yeah, it's like April Right right about a year So we can expect rel 9 let's talk about in-place upgrade because um, yeah As we talked about already in this episode right rel 7 is Getting to be a short timer and Rel 8 rel 9 is is where you're you're wanting to be So tear i'll be on in a next next episode and a few episodes past that um And i'm gonna be better about booking guests for short because Oh, you're fine. Did you not like just the one two tandem? I mean just the two of us just the two of us So it was so lovely Indeed yeah indeed. All right, so cool Uh, yeah, so folks. This is it for streaming for today I appreciate everybody that's tuned in throughout the show throughout the day today You know stick around You know subscribe to the calendar if you have not already We have a calendar of events that you can subscribe to and know every show that's coming up in the future Including brand new ones that we're launching this year We have one on dev sec ops coming up and we have an eu based show That will be a morning coffee break kind of show for the eu so I will be asleep And the show will be happening. So yeah, stay tuned and uh, you will have more fun red hat things to watch So thank you, and we will see you all next time. Stay safe out there