 Good morning. Good afternoon. Good evening wherever you're hailing from welcome to another episode of the level up our I am Chris short executive producer of Openshift TV. I am joined by the one and only the illustrious Langdon White Langdon. How are you doing today, sir? Not too bad. Not too bad Other interesting show today. So that's what we're doing pretty well. I wore my Octo Thorpe t-shirt Which if you are unaware that is the original name of the hashtag Because if you look at it, it actually has eight crosses in it So the Octo and a Thorpe is the little right XE thing or plus I guess All right. So there's your etymology lesson for today folks Yes, the little shout out to the TV show on the CW called the flash Who has a character? So I used to watch with my son, but there's a character on the show who wears Very unusual t-shirts in basically every episode and this was one of them and My wife bought it for me as a present one year. I could do that job. Yeah. Yeah, I can wear I could wear cool t-shirts Yeah, just wearing cool t-shirts all day. That would work It would not surprise me although I am unaware of its existence Entirely, but it would not surprise me if there is not some sort of fan page That is just all of the t-shirts and like where to acquire them and they're mostly pretty good So it's worth it's worth taking a look But yeah, so we have Yeah, we're gonna talk about sub sub good sub G. U. I D sub IDs. I don't know how you actually say it You know, it's one of those, you know, they pick your poison, right? There's no right or wrong way Yeah, I was in I was in like a research call yesterday and they were talking about basically a jitter inside of the data science work they were working on and They said the word perturberate Can you say perturberation? Which is P P E R T U R B A T I O N Okay, which I bet I might even I'll even throw in the chat so that I can Perturbation which is a word. I bet all or many of us have seen before I have I don't think I've ever heard it out loud Which I thought was kind of funny, but it's basically like it means like the how the fluctuation works kind of ah Where the fluctuations are the action of perturbing? Being perturbed nice. Okay. Cool. Yeah, that makes sense a disturbance of motion course arrangement or state of equal Yeah, see So, yeah, it was just it was one of those things where it was a serious throwback to when I was like Early college where you know, I knew a lot of words Primarily because I read a lot But if you read a lot, you don't always know how to pronounce them. So like K versus key Q you a Y Really threw me both that I originally thought it was pronounced Quay Because it looks like it is then eventually I learned that it's actually pronounced key Then I met the team who does the product that we ship called that That is also pronounced key So it's all very confusing But yeah, nice. All right, so there's some etymology and some random pronunciation Let's let's hit those slides to get back on track. Oh, I was about to say let's let's get the thing back on the rails here Yeah We yeah, it's it's like I was It's like the third rail except, you know, I heard a really good quote about that actually he's like, you know You often will hear people reference, you know, don't do that thing because it's like touching the third rail and I Also, but I heard kind of a counter to that which is like Yeah, but that's also where all the power is And so sometimes it's that's amazing. Yeah, isn't it really amazing? So to attribute the quote it's from the TV show the West Wing Okay, fair enough season five maybe I think the fact you even remember that is impressive Well, we I've been rewatching it actually with my oldest son who's very into politics And so my wife and I and he have been rewatching it and it's Like I don't remember like any of the episodes my wife remembers like every little detail about every episode And my son's enjoying it. So we're we keep watching it. But yeah, so attribution, but I thought it was a really good line But I'm a big fan of sorkin in general, although I think by that point he was not actually writing. All right, so slides Um Yeah, this is going to be a heavy tangent day. I can feel it already. Yep. Yep. So the level of our that's what we're here for So a little bit about what we do here. We talk about generally speaking we talk about containers and Kind of things that are related to containers as well as like things like orchestration of containers Primarily with a focus on this is why you should get into containers they're super useful for basically anything you do in technology and You know, they're they're basically a nice self-contained little unit to do something in It was actually really funny when I was preparing for the show yesterday I tried One of the things that I was trying to figure out But you know container and then I did a different operating system container and then I was like I don't know if these containers are actually set up the same way as a virtual machine for this thing So I actually spun up some virtual machines And it's it always kind of bothers me when I have to spin up a virtual machine to say something Because it feels like I've dropped the ball somehow But it suffice to say I use containers for everything and recommend that you should too So a little bit more about the show you can find myself on Twitter Langdon with a one you can find Chris with Chris short with two S's right there in the middle and You can chat with us on our discord. It's been super quiet lately But it was pretty busy for a while and then it's been really quiet lately So, you know, I haven't been dropping the link as much so maybe we'll just do that here now and perhaps perhaps I've seen I've definitely seen joins but the the chit chat has has died down a bit But you should definitely come and hassle us there Somebody you just said quick time in the chat which scares me No, we're talking about pronunciations Yeah, QT. Yes. Oh, that's funny QT. I never really thought about it as a quick time I'm pretty sure QT is is meant to sound like QT In the men too, but I will never say it that way. Yeah Not quick time, but the no framework for right drawing stuff. Yeah So QT gas station apparently really I Was funny. I was actually I saw an article the other day about how some Like coffee and not coffee gas station chain was kind of coming into the south from like Texas or something I wouldn't say it was called like beaver or something. It was something like some small animal I thought it was beaver, but I was like I Went and read the article cuz I was like, I don't know I don't even vaguely understand what this thing is about from reading the headline Because I'd never heard of this particular convenience store Which is funny just because you know, I know Wawa even though we don't have those here You know, and what was the other one the other big one? There's another one that people like swear by and we have but we have like come bucky Buckies, that's what it was. It's always your logo. Yeah, that's what it is Yeah, so we're that's like seven tangents in eight minutes. I know I'm doing well Yes, which I'm familiar with as I used to live in the south. Yeah. Yes. Oh, actually, I remember that one. That's funny Okay, so You can find out more information about the show at red dot ht slash level up our and then this is episode 36 and we're gonna talk about subordinate user IDs. It's a little broader in that but seemed like good shorthand and The notes from last time are at episode 35 and in a minute, we'll mention that a little bit more just because it's funny And I think that's the last of our slides before Yeah, before we get to the point so we're going to stop there So today we are gonna talk about user IDs Okay, so and kind of to start to kind of head back to the basics a little bit as Many of you may or may not know one of the defining kind of characteristics of Linux and by extension Unix is that It's kind of a loose term, but like everything is a file Okay, and everything everything everything is a file. So like a socket to file, you know You know your USB drive is a file, you know, everything's a file and what that does is Simplifies the operating system and how it works in a lot of different ways I'm not sure what originally kind of caused the designers or whatever and of Unix to kind of latch on to that But everything's a file. This is majorly distinct from Windows where I'm Different types have like or different things have very different types So if you are a longtime Windows user user, you may recall that, you know a sim link and a shortcut are Very different because a shortcut in Windows land is a very unique weird little thing Whereas a sim link is just another file. So you have all these different types of things Arguably when you have lots of different types of things you can make each of those individual things Perform differently under different conditions, right? So you can say, you know a shortcut will do things slightly differently Maybe but with us like a sim link, for example, that's much harder because it's just a file, right, right? So so there may be good reasons and bad reasons, but it is a very major distinction between the two And so if you are very familiar with Windows and you come over to Linux It's something that you really need to learn because it can be really confusing About why, you know, like why everything is a file So that's kind of the first step of the conversation Next what we talk about is okay now we want to defend One file kind of from another or you know or like one user from another user And so what what was done was there is this user concept And each user has access to files, right? And so because everything's a file, it's very consistent However, different users obviously can only access different files and then you have kind of three major categories of file security, right? You have the ownership is by user by group and by and what's called world. And so if a file has Set is set correctly only yourself can see it, you know only your group can see it And then a group basically just by way of kind of finishing the explanation a group is just exactly what it sounds like a group of users Is this why you were complaining about Linux groups the other day? No, actually Okay, nevermind. No, it's kind of funny though. Actually, I should bring that up because it drives me crazy No, but I will I will be complaining about the nobody user in a minute. Oh So so yeah, so groups They they really in some ways feel a little bit like an afterthought But I you know, it's kind of one of those things where we see in in the software world You know, we're in the computer world kind of more broadly Sometimes it's a little hard to tell whether something is there by design or it just kind of evolved for that way and so You know and and because the industry has been changing so fast It can be it makes it extra difficult So one of my complaints about groups for example, and I'm sure anybody who's done any like Linux administration will have this pain, which is that if you put a user in a group That user has to completely log out and log in again before they get the group back And so you can sometimes short-circuit this so particularly, you know as when you're kind of administering your your own user By essentially just running bash again But if you are in most desktop environments, so like no Mercady or whatever this doesn't work So even if I am in a terminal window and I run bash it will not reset my groups to the new set So this does sourcing the profile work. I don't think that does not in like a GUI No, right. So so like no Mercady or whatever. Yeah, like you have to log out and log back in where Yes, you can do like SSH local host or bash or you know Just right run a new session of your shell and you'll have everything right there But essentially you're now running two processes to do the same thing exactly So that's that's exactly it and so it's a real pain in the butt especially and what happened to me the other day was I was Setting up I can't remember what I was setting up now But I was using ansible to put a user in a group and I did it wrong and I made it their only group So I took away their other groups instead of Yeah, yeah, you forgot to do a pin, right? Because that user was who I was logged in as and because I was logged in as that user it wasn't changing the groups and so Yeah, so I ended up Borking the user and having to like rebuild it and so I was angry and complaining to the rest of the team about Why are groups such a pain in the butt? So long story short? Yeah, and so Rex roof makes the point if I change my users groups. I just reboot So that's the thing right is You're on a big server with a lot of users are running your application. That isn't exactly exactly Boot is not pretty Even even if you're the single user on a proper like data center server. Yeah, that's a 10 minute operation Right, like if I restart the you know the cluster in my basement It's five ten minutes before that thing will come back up, right? And that's wicked fast compared to ten years because it's just me. Yeah It used to be Make a cup of coffee Right, right get and then come back. Yeah. Yes to JP Day's point never reboot. This isn't Windows You know which I don't know like some of the nomen choices over the past recent years I find very odd in this regard because they've actually set it up so that the expectation is that you reboot every time You do any sort of upgrade which is just kind of like the opposite of what you expect from a Linux Kind of distribution. So well, I'm sorry short there, you know You know, I didn't want to reboot because I had you know a million windows open and I was kind of in the middle of something And so but I needed to be in this group for whatever it was to happen And then I ended up actually working the user. It was this whole thing So suffice to say as of Memorial Day my primary, you know, my as they refer to in the phone world my daily driver It has been upgraded to Fedora 34 because I ended up just wiping the entire machine Well, I somehow it was all about virtualization. So it was all about And I somehow broke it badly Wow It it was badly enough that my kind of quick and dirty fixes weren't sufficient So I was like whatever and I had to do the upgrade anyway So, you know, it was I was like, yeah, I'll do it now. I've done upgrades in place. I'm for the worst since like Yeah, so I'm a fence surprisingly well, yeah So it totally does and I highly recommend it. The only reason I don't usually do upgrades is because I'm I'm very close to the desktop Experience and all that kind of stuff like like I'm on the the working group for it in Fedora land You know, I I try to push for developer and sysadmin stuff You know, like proper sharing exactly and Actually, it's one of the things on my list to do is actually to try to move all of the Fedora like workstations Even server over to RDP and just get rid of the NC altogether not CNC VNC and But So as a result, I tend to do a wipe reinstall Because I want the brand new experience. So I want to know what it feels like when you first started it up You know, I want to not have I don't want to stomp on this on the new settings. I want to see what those are With my old settings. So if it's really a daily driver and all you're trying to do is get the latest version of stuff, right? In place upgrade highly recommended works really really well. It's super fast But you know, I just kind of have an unusual use case. So I tend not to do it now, but yeah, exactly all right, so Groups users own files And so to kind of continue on that same vein The nobody user was introduced Quite a number of years ago now. It is actually an official designation in Actually kind of by way extension. There is this I always get the letters wrong. I think it's the LSF the Linux No, LFS the Linux file system standard standard and Why is this interesting? Well, so if you go back to that remark that everything's a file Well, the Linux file system standard would therefore be the complete definition of Linux essentially And just as an aside, if you're not familiar with LFS, we talked about it on the last episode of Red Hat Enterprise Linux presents. So let me grab that real cool. All right And you know, it's it's much like RFC's Where I'm not finding the LFS Like link to the actual thing just it's actually surprisingly hard to find Unless you'd like type in Linux file system standard, you're not gonna get it, which is annoying That is weird Yeah, like I found the Wikipedia page, but yeah, so we have that but yeah, so Here it is. Hi article file system. That's FHS. That's weird Yeah, that's maybe I think that's part of why I can never remember which way it goes Because it's the I think that's the right way. That's what we use. Yeah, this is what it is last time. Okay. So yeah, this is the right thing I mean, well, they obviously thank you, but I think this is it Which means it has changed in name It hasn't it's just weird. Yeah So, yeah, so if you go to free desktop That's where I don't know It's just one of those things where it's a little bit hard to find if you've ever tried to go find an RFC That's not trivial either. Um, so actually I'm really good at finding It's because I've had to tell so many vendors they're breaking them, right, right If you could show me the amalgamation of how you can ratify that an email address is valid Using the I think it's seven different RFC's that all conflict And are inconsistent Yeah, there's basically no way to actually determine if an email address is valid as far as I can tell so Yeah All right, so long story short In that definition there was this user that was that was conceived of in a sense called nobody and has an associate group also called nobody which is The user and group that a file is owned by if that user and Excuse me that user and group take a drink or something real quick Yeah, right that user and group is not present on the system is the way it was defined got it so oh And I'm sorry whoever said a man here it Rapskallion reads again with the awesome commands. Yes, that is 100% valid command and it tells you in Succinct detail what each directory does or is supposed to do on your particular system the man hire. Yeah As an hierarchical standard, so yeah pretty cool How interesting? Yeah, and I bet that is different depending on what operating or what flavor of Linux you're on Oh, because this is where some of the distinctions come in Which is like one of the biggest challenges between the different distributions of Linux Is actually significantly more about muscle memory than it is about actual differences And so knowing where a file is is one of the things that your muscle memory tells you And then the other one is kind of like how do you install or update software? You know on the Debian flavors, right you use something like apt-get, you know and on the rel and Fedora, whatever you use something like yum or DNF and Why you know why the the differences are there? A lot of the time is just because you you're used to one or the other But so there's this nobody nobody user and the original idea behind it was that you're so you have what are referred to in Linux land and formerly Unix land I Usually pronounce it as Damon, but technically speaking if you read the like letters, it's actually demon It's I think a little bit of a joke But the idea is that basically things that are running in the background or things that are running all the time that you You know don't interact with you're not attached to or whatever So the idea was that those things could run as this nobody user So then if they were compromised you wouldn't be able to get anywhere Because you didn't have any permissions like general permissions on the system Challenge was is that as soon as you had two of those daemons Running as nobody then they can access each other, right? So that didn't work out terribly well so What you ended up seeing is the now you have this kind of nobody for almost like special use cases and you have this and then you know like Apache will have its own user Right or engine X will have its own user. There are still some daemons that run as nobody the example in the articles I was reading about were NFS for example But they're getting They're getting less and less common because of the problem of you know If you if something is owned by nobody then anybody anyone logged in as nobody can you know get to it Yeah, you know you can't log into it as a human But if you compromise the NFS bridge for example, then you could access, you know, some other thing that is owned by nobody So all right story short for that when we talk about it in terms of containers The reason I bring up nobody is because in a container if you Don't have privileges on the host system files that you have bound mounted into the container the way it Does that without you know kind of breaking everything in Linux is by assigning the ownership of those files to nobody The thing that I don't understand about this and this is where I got a little confused And maybe I will show you now DNS mass runs as nobody that's a good point. Oh it does. Yeah So like I said, I just saw and the thing I was reading was The example is NFS. I didn't look too hard into other ones. I mean, I know there's definitely a few I remember seeing it around I just couldn't remember what all they were off the top of my head So let's see Sorry All right, so as I said, I rebuilt my machine. So some of the settings are new and special That is not the correct window Where do we go here window? this one all right, and so the quick and easy example of this is Let's see If I get it in my scroll back, I do not so Oh, I think I have a cheat sheet. Hold on a second. Let's see. Oh, yeah, here we go But we'll just do this slightly more interactively than they have here Oh, so this this along with a bunch of other stuff is from Various articles about rootless pod man And I will put them in the further reading on the show notes rather than kind of dropping There's at least four that I have here rather than kind of putting them in the twitch chat But yeah, some commands in twitch just don't work. Yeah, exactly All right, so as you see here and let me know about the color Lightness size of far. I think it looks good, but folks. Let us know as well. Yeah, I try to see it Yeah, I think they are correct, but so are you using Wayland and pipe wire? I am not using Wayland. I might be using pipe wire. I don't know actually, but I'm not using Wayland Primarily, weirdly enough my my primary use case for not using Wayland is Because I like clipboard history and it is still not built into gnome Even though I filed a bug about this really? Yeah and There but there is an extension for gnome that will do it and supposedly it actually works and this is something I was gonna experiment with it supposedly the one I have found recently works actually with Wayland I just haven't tried it and I also want to I need to experiment With like zoom and Google meet and all that stuff and see the OBS just got an update yesterday So it works in Wayland, so I'm hoping okay zoom does actually I think zoom did get updated so it works in Wayland now Okay. Yeah, my other primary use case. I hardly use anymore is Shoot what's it called? There is a tool in Linux that will Take two computers and let you use like the two monitors, whatever, but it lets you use the keyboard and mouse going across them both It's so it's kind of like reverse KVM almost Not KVM the virtualization but KVM the you know the support connective computers. Yeah, yeah, whatever But yeah synergy. That's it. So synergy last I checked Also didn't work in Wayland and I was using it a ton But I am not using it very much anymore. So I may that may be a reason to stick with it So Okay, so what I did here was I mounted a directory into this container that should obviously have a bunch of privilege stuff And a bunch of stuff. That's not privileged so if I do and As as per the docs it was also talking about this You're gonna get a lot of errors doing this. So I'm just gonna dump it to dev null and so All these files are right and so that's kind of what you expect. So nobody's Nobody nobody's So let me just see So actually let's use sudoers here, right? So or sudo cough, right? and So that's scary Right, exactly. So if I do tester and then sudo com And I get permission to night so that's cool, right? So Currently right and you can see in my my prompt but just in case right? I am doing I do a who am I? I am root inside the container. I do an ID. That's how I'd find out what my you ID is and my do it Go ahead not good idea So You can see I am you ID zero and I am G ID zero so therefore I am the root user on the machine one of the things that we didn't really mention but root is Not only special in that they can access all files But the way they do this is a little bit of a hack in that they are ID zero and G ID zero This if you look into C groups at all and we talked about this bit on the show with Dan Walsh whatever some weeks ago and Basically that that root power being tied to the number Has resulted in things like providing capabilities or what are called learnings capabilities or sorry colonel capabilities, I guess Made them a little weird because there's a lot of things that just check for that number And then allow the operation and if you have that ID number then you can do it But so this has caused obviously as you might imagine some problems where you only want to give the user certain capabilities but not others but they also happen to have you ID zero so well, I'm sorry short there the the point being that This root thing is also a little weird kind of like this nobody user and this goes back to Computers been around for a long time now, and they've evolved a lot as they've gone So you've seen a little bit of evolution in this root user that was not accounted for in the original design in a sense So yes, so my point in this sense to WSL course there maybe is a You know this route is not the same as the external route and we have another cool little command Which will show that in a second, but long story short what I wanted to show off was that if I try to read this file I get a permission and I'd even though I am root user Now this is where it annoys me and where I'm going to ask some people on Twitter To explain this to me because if I leave my container I Expect my Nobody interactions to work the same way, right you know, so Just to show you I Was fooling around with this yesterday But I'm gonna do it again just to see just to kind of show you what I was doing so if you do pseudo and minus you you can provide a user and I can let's say make a direct because I thought maybe it was Excuse me God So if you notice when I was doing this above here, right? It was under this tester So I thought maybe you know basically it was being able to walk the directory or something that was making it so that I Was getting permission denied because if I do actually we'll start here Let me do Touch Tim oh Unless it's because it's in temp. No, that shouldn't be it but a favor and like Clear shouldn't like move this command up to the top. Oh sure and her a bunch or clear or whatever. Yep There we go, there we go So, so I'm just gonna let's call it that and type in my password and then we're gonna say temp and As you can see Everybody has read access or sorry to everybody, but so I have read and write access My group has read only access and the world has read only access So we're gonna take those permissions away or most of them By doing zero six zero zero. So this is an octal set So, you know if you put in a four that's read if you put in two That's right if you put in one that's execute. So if you add those numbers together That's how you get the number that you put in here So this will give me read write on it and no privileges nobody or The world because I nobody grew Which should be fine because I am currently I'm a white right so my I have this group Oh and actually in ID. I don't think I realized this but you can actually see the other groups on part of So I am not in the nobody group, right, right So I know you're you're a nobody to nobody exactly So now if we LS it again, we'll see Here and we'll do this again. Thank you. We'll see that it's only Read write for myself should have no other permissions, right? So now let's just put something in it by doing nobody echo Some text in the file and we'll do Tim Really Why is this not working? Oh, I know why The the Substitutions, I think get a little wonky. Oh, no, no, it's not working for me Did I give myself right? Yeah All right, so of course it was my experiment going to be totally foobar now because This was totally happening for me yesterday All right, and so let's just do the LS again. All right, and then All right, so we have that down there and then so this is where it's kind of interesting So I did this even though I did a minus you as nobody it actually created it as me Which I thought was odd But you may have to do with substitutions I'm sorry, did you run a like do you need a dash C is what wraps going and Reeves just asked in chat. Yeah, I was wondering that too No, oh Sorry, not the echo. There you go No, it's dash C is it wait I Thought it was that C to but is it s H dash C the not s H eat the dash C Yeah, no quotation mark in front of echo because that's the command and Then you need to quote the text. I Thought that was what I needed but then it wasn't working for me. So that's why I was playing around with it No, see it doesn't ever Man pseudo, let's see dash see coat but Raps guy and Reeves as quotes before the chompy thingy Like this, I think it's because we need We need sh or bash C You might be getting pseudo and SSH mixed up. Oh, that's entirely possible Yeah, there we go, there you go Hit enter a couple there we go Yeah So, yeah, so eventually. Yeah, I need the bash minus C and now if I oops If I do LS dash L So now this is owned by nobody. It has some content. So this is what we wanted, right? So Change mod zero. Oh need to pseudo change mod zero six zero zero Temp five All right, so now Only nobody has read write on Tolua five and we know it has some content. So as Me, I shouldn't be able to touch that Seems reasonable, right? But if I do pseudo Mm-hmm. I also shouldn't be able to do it because going by If we go way back here Mm-hmm To inside the container Yeah, that's right. I failed last time, right? So we did a cat as root We can't run pseudo in the container because pseudo is not installed, right? But permission and I right? Weird so What I did was I was like, okay, so maybe it's the directory and so let's say new dur and then we're gonna Sudo shown nobody nobody Temp new dur and then we're gonna pseudo Change mod zero seven zero zero that's a lot of zeros Oops, I do that all the time Okay, now if we do temp we should see new dur here No one has permission on it. It is a directory and we have execute, right? So on a directory you need execute permission to be able to go into it Yay. All right. So now let's do our command that took us like 87 tries Rounding exactly And then we're gonna make it a new number just to make sure we don't screw it up All right. So now in theory Actually, let's do minus you Ah, you can do it. I believe in you All right. So there's a file there. It is got a lot of readership. So we're gonna we're gonna take that away Just in case even though we shouldn't be able to get there at all so zero six zero zero on Oops, what do we call it? Oh to you and tab completion doesn't work, right? Because I don't have execute So I LS it again. Okay. So it should be just rewrite and we LS temp and New dur is also got no access to it and then if we So let me try cat New dur H6 Great, that's expected. Great. All right. Let me throw pseudo as desired. Hold on Not Understand it's not supposed to work so I need to ask the internet because I don't like like the whole like that whole story, right was about Using the nobody user inside the container makes it so that we can't access it from inside the container using root or any other user But that's not how it works on the host system, right which I was like we what? so Isn't this just due to the namespace root i.e. if you ran podman as root with pseudo you should have the ability to read a nobody file So maybe that's what I expect exactly So let me see people are asking questions and they're like why don't what yes, right? work What's going on here? Right and so this but you know kind of what we're talking about here, right is the rootless hot man thing So let's run this thing actually let me throw a dash. Oh, no, I do have one So we're gonna run this one with pseudo so as root So oh shoot, sorry, I forgot I don't have it downloaded Yeah, exactly It's not even that it's just whole all right. So now in test there should be the XC at C directory So we'll test der and just in case we'll throw this we don't get a huge wash of error messages But so the file we were much in with before so if you notice it's now owned by root and not by nobody Interesting and that's because you logged in as root, right? So now if we do cat tester Or you ran it as rude I should say not logged in but yeah, right Pseudo.com right so that seems fine. Okay as expected So so you can run it as root and you and then all the file permissions are set for the basically root ownership And I should be able to actually modify those files Is there something actually I could just touch something and that would be enough of the test just To Oh, wait a minute, so I'm running it as root and I have the directory, but I don't know if I have directory permissions or if I just have file permissions What's the do a LS LD on the directory itself? So test der Yeah, I don't know I have right yeah, I'm executing everything so so this is where Yeah, so where this is where some of those permissions I get a little hazy on Because I think is there anything in Etsy I can mess with without actually breaking my computer Let's look at I should be able to just add a pound side and do let's just do pseudo conf because if I just add some oops If I just add something to the end here Yeah, so changing a read-only file Test of editing Yeah, and I cannot write it. I don't know if you can see that at the bottom of the screen But I get an error So I'm gonna exit out of that so I can't actually modify these files But I can read them and this is where I start using like colon Z Which will do the the mounting Basically, it'll match up the the user ownership So yeah, so it's all very weird It's all Linux madness exactly exactly So so W I mean it's fair he was confused about where we were previously it Worked as desired in the container originally it did not work as desired on the host OS And now it's being weird and So now we're back in the container, but we ran it as a full container And so it's you know working a little weirdly. I do know that if I do this It will oh, I know why it's because Yes file system right so the file system permissions are Letting us through but se linux is not so if I so in theory If I do pseudo set problem pseudo set in force Zero so turn off se linux. I Yes, I you're making dan walsh cry as we speak right Then if I run My same test I Won't I bet it will work And pseudo Look at that Test for And look at that. I was able to write it and if I now check that file Let's see pseudo If I can type that file Good I don't know if you can see it at the bottom here, but so yeah my my change took So you have a bunch of different things interacting there, right? But yes, I don't know why this didn't occur to me earlier But se linux so while I have So as you may or may not know we talked about this on the show with Dan Se linux is kind of like an add-on security layer onto Linux, right? So this is why for example I think it does a boot to a bunch who still doesn't have a sea linux right now. No So a boot to for example doesn't have a sea linux because it can because you know, it's it's an add-on They have a different tool That I think it's called app armor that does yeah, you know same concept different way, right? so It's an add-on which is then applying a secondary level of security on to our file system And when I do set and force zero that turns it off, but we'll turn it back on to make sure that Chris's shirt is okay shirts be safe, right, so By long story short when I have it on Then when I'm inside the container it's se linux. It's blocking it. In fact, we should Shouldn't we get a denial even Wait, no, if I do it without that it'll help be less automatically and then What is it called audit probably is I'm gonna have all my pseudos in here, too I went to write Just looking to see if I can see it in here quickly so in theory in here there is an error message That says we didn't let you edit this file because you are an idiot and you shouldn't have tried to modify it. I Think it's probably nicer than that, but you know you get the impression. That's what they really mean So actually now I'm curious this Let's say control Semi-colon no no journal dash xe. Yeah, no, I was looking for how do I do you split pain in T-Mux because I can never remember I need a T-Mux sheet. Yeah easy for me to say Splits or no vertical split. I want I got it. Yeah. Yeah, all right, okay So in theory actually I do and you too can get this wonderful Cheat sheet that I have for T-Mux from open source.com. Oh nice. Yeah Check out all the cheat sheets are really good There's also. Oh, here we go. This was before Resize pain She's doing all kinds of T-Mux arey today. I know well, it'll be worth it Resize current pain Control be control up down or control left right Control wait, so control be then conti. We got that's what I wanted. There you go Ignore the error behind the curtain So now if we do journal control Minus be is there something you think we can oh what's what's the errors just look for errors You just said x e Yeah, just sexy journal cuddle dash xe. I think yeah. Oh actually Let's do minus f All right, so What do you think is this gonna work? Now we're gonna do Podman run my T RM and Let me just grab the minus V so I don't screw it up And fedora and then Vin bash. Yes. I'm sorry to laugh at your pain But Narendra have just said today's T-Mux circus show performance fired by Langham queue up your applause Don't even start don't even start All right, so I mean this is why I've kind of walked away from T-Mux on screen right like I will just background a process Before I open T-Mux I wanted to see if we could actually see the error in real time when we tried to do it right so that you know There's not a lot of I didn't have a lot of choices there. I could go horizontal. Maybe that would be easier But this should work. Let's see what happens. Um, oh, that's reassuring. It's not even there. That said permission denied We were able to wait everything's in tester I thought right Right. Oh, oh, no, I'm supposed to you're running his route. That's the problem. Oh, that's yeah See what all kinds of output now, right? All right, Narendra says he likes his T-Mux with fish shell. That is I know I know he does I So I use T-Mux successfully very well with fish shell to be honest with you There's a Vim prologan that opens debugger output in a separate T-Mux window and I love it What is that plug-in because I would like to see that Okay, so let me go over here and All right, so that's good so all that stuff was basically us, you know doing pseudo to run it, you know, whatever so But in theory now if I try To let's Yeah, more stuff And then Well, that's interesting. I don't actually I get the right on the append. I Immediately get the error and I don't get the error when I try to actually write it Which is weird But whatever that's just how vi works. It's not related to what we're talking about But as you can see it was blocked by se linux because we tried to access it And we do not have rights on that thoughts Q bang sorry and then Let's just do this. Oh, yeah, and then How do I kill the pain is X? Okay, so then back to this Spot here if we But I want to show now so so we get the Se linux denial But we had permissions on the file right when we did that When we just did an SL at an LS we just looked like you know, it looked like we should be able to edit it but we couldn't Because we are root and yet we cannot and the reason is because se linux was blocking it underneath Oh, that's what I was gonna do. So if I do LS dash LZ So capital Z shows me what se linux settings are on it It's a system you and if let's just say for the sake of argument Let's say mind mount Thing and then we will copy we can even copy pseudo.conf into bind Mount thing. Oh, I thought I could do that And then let's change the ownership pseudo tune. Oh, wait. Oh, wait Find mount thing Main risky says time to make an audit to allow module to let you edit that file exactly I think it's probably a little bit more than I need So now if I do Let's mount Actually, I'm gonna need I need to find the right buttons And Mount Thing I should have done MNT and then I would have been able to type it a little bit better So we're gonna mount that into tester which should give us access to it We're gonna use the colon Z to tell it to relabel. Oh shoot, you know what? We're gonna have to see so if we do B percent Let's look here and we see bind mount thing. Let's look at LS Z bind mount thing If you notice this is an unconfined user object here in user home T And if I look inside the directory. Oh, that is actually inside the directory And then if I do this it's basically the same but when I do Let's see here. We'll get cool about it What is it watch minus N1 like that? Actually, let's do inside the directory All right, so theoretically we'll watch that change and then if you notice it changed being a container T file So see Linux has relabeled it and so now if I look in tester pseudo conf so This is one of the things I think is super interesting, right? So we just got done talking about the nobody user, which is supposedly the user that is used when the ID doesn't exist on the system However, this ID does not exist on the system. That's why it's showing me thousand thousand, right? So, you know, if I do cat at the password Thousand isn't here so because it so it has no name, right? But it doesn't make it into nobody nobody which You know this this whole thing clearly this whole thing is like weirdly explained, right? So it's almost like we need to bring on somebody like a Scott Yeah, so maybe just be like explain how this this works. Oh us maybe Matt machine can tell us or you know Anybody right like yeah, why I will say when I proposed the this show originally I did say that We should really probably bring somebody on for this because it's gonna be weird but Look at that. I can modify the file because it was relabeled. I'm over here pointing which I know isn't helping anyone Right, but you know, so it was relabeled as a container file T So now I actually was able to edit the file and If we do oops No Not that one. I want to kill this one So yeah, so if we really have a little whatever so, okay I can't believe it. We're starting to run out of time. Yeah, this has been mostly about user IDs and GI Like just regular old user IDs and group IDs But to talk quickly about the or should we do points and then talk quickly about the The Yes points. We do try to get them within the hour. Yeah, people do have to duck out. I'm sure at 10 o'clock. So Skip those meetings. Yeah, you know, tell them Chris short said you could skip the right exactly All right, so points for today. Oh I do I Rebuilt my machine so I had to redo or so I had to put back together my tools that do the calculator points So if you see an error or explaining the point system to me last week, I feel like that is prone to error It's it's pretty automated Basically the the challenge was just I had built a container that runs the go get the spreadsheet from Google cloud But that container had a bug in it it was missing a library But because I already had it it was one of those things where you know, like I didn't do like a clean test on it Yeah, so yeah, it just needs a little bit of editing. I think it's fine now But you know just in case So as you can see in a rendev has an awesome 5,600 points that is I'm gonna throw the code in the Chat Sorry, and then Netherlands hack them with 5,300 points. No friction I think it must have missed last week is only at 4,000 points Joe Fuzz is still holding strong at 2,300 points But detective Konokudo is up and coming as with bacon fork 2,100 and 1900 points respectively And they are both ticking away, you know, there's this is the first time where the leaderboard has all been over a thousand points Or close to 2,000 points Yeah, yeah, okay, we're getting up there. Yeah, it's pretty wild 36 episodes in there. Yeah Yeah, it's kind of crazy So, uh, yeah, we we're really excited that you generally participate in these they're they're kind of fun And we got word this morning That one of our legal hurdles has been Surmounted surmounted. Yes. Yes, eventually still soon hopefully Right, right Stringsic value will become intrinsic value other way around Intrinsic value will become extrinsic. Oh Jesus Right. So tick tock tick tock tick tock You know with any luck Chris and I will actually see the swag before we are sending it out to all of you for your points Maybe um, but we are not holding our breath. No, so you'll probably get it before we do somehow. Yeah By the way, this is gone and What I think is particularly funny about this is actually redhead in general when it comes to like swag and giveaways and stuff is Actually one of the better performing. I don't know what the right word is for this In that I've worked with kind of in the industry at all ever in general. It's really really good and oh Yeah No, I don't want to have that discussion today Kind of final note You know like this is really anomalous and that's why it's really particularly funny for us with the show is that it's so unusual and so I'm surprised that Chris doesn't actually have a block to filter out anything related to IRC Just in concept So yeah, I liked IRC when it was new 20 years. Yeah, oh when it was new before that like But like in the 90s it made In 2021, I'm sorry when the competitor was I seek you and aim, right? Yeah, like Not that there's better options, but there's more options, right? Like I Need to be able to send those Giffy's I know right like I I like my messaging things with the ability to send images, right, right? So what was about to say so yeah, so points. Thank you for participating. We hope it will be You know as as Chris said as I often say turning into extrinsic value very very soon But we we essentially we I think we've actually gotten over the last legal order. We just have to We need to kind of execute the content. Yeah, just publish it It basically landed what this was this morning, right like 7 30 this morning. Yeah, I was like I couldn't remember who 7 30 this morning 7 30 yesterday morning So yeah, so very very close. We really are this time But that's the points going back to our Regularly scheduled programming for just a moment. I wanted to kind of cut to the chase a little bit like like What I was trying to give was the context of why That was the wrong window Yes, it was it wasn't anything useful but But it was the wrong one I had grand plans to that being the right one and it didn't work out that way the rundown is Freaking out over the fact that people still use IRC. He just discovered what it was But there's a whole story there there's Actually, maybe I should I could throw the our second article in there But there's a there's a whole drama going on there right now, but IRC is actually a heavily used place for Kind of the Linux open source community in general, you know, so like Kubernetes for example uses slack but you know, no, they actually have their own IRC server as well as they do a lot of chatting in a free note, but whatever The but the long story short day if you if you want to participate in an open source community a lot of them are in IRC And there are like many there's many IRC networks to choose from right And there's many IRC isms in the sense that like anything else in software if it's been around for a long time there's very strong and strict is kind of the word I mean but not quite Protocols and methods and like usage patterns that kind of stuff, you know, one might just call it manners in a sense or Is that what's the what's the like proper term for manners? I blank in the word But there's basically this whole set of rules that you should follow when you are using IRC Make sure you know what those are what's in my opinion the fact that there's Things like that whatever well, I got some to there. They're certainly evolving. Yeah, I need a kit They're thank you reps guy and yeah, they're the fact that there's a net a kit that everybody has to learn before you even join Or use something that disturbs me. Yes. Yes, right? You do get a little bit You know of grace period in a sense Better than some it depends on the channel. That's true I have gotten kicked out for asking any simple question from IRC channels before that's documented get out, right, right? That's super friendly. But yeah, yeah, so long story short That's the story behind or some of the story behind IRC. There's a lot more to that story, but I Don't remember why we were talking about IRC, but to talk about what we were talking about for the show I will in the further reading on the show notes I'll put some links to some good articles about this But the reason we were talking about it is because occasionally you will run into Did I pull the link where to go? Occasionally you will run into There it is There is an error that you get which I'm just gonna try to grab it Yeah, good God That this Do I need to put on the music? So I just put it in the terminal window. So Occasionally you will get it that error from podman What this means is that You're basically inside your container. It's trying to use a bunch of different users To accomplish it even just one but it's trying to create a user in the container that has a particular ID If that user ID is kind of not available for a large number of various Different reasons it's gonna throw an error about trying to let you have access to that ID so the way that podman tries to fix this is by mapping those IDs Basically two IDs that you have that are what are referred to as subordinate user IDs So while I have my ID, which is 1000 on this particular computer Let's say I want to have a range of IDs that are available to Inside the container so I can have different users within the container and so, you know, just kind of by default if we Hopefully if we do Let's do the other one Sue doesn't really matter But I'm hoping this will show What I mean Not so much I can't think of any great examples without like installing the stuff But the point being is that you'll log into a container and there will be a bunch of different users They're owning different files and stuff So what you need is a way to map those users to like real users on your machine on your host machine If you want to operate on any of those files or any of that stuff So in order to accomplish that goal with all of the noise that we've just spent the last hour talking about They've kind of created this concept called a subordinate user ID and one of the things that I found is that Running into that particular error or errors like this should be getting increasingly unusual because when I I'm doing this right. Yeah, here we go So what I discovered is that it seems like a lot of the Distributions are setting up this by default and So what this is saying is a set up this range of user IDs as subordinate IDs That's what this brilliant sub you ID is here So that they're available to my user however, that's Excuse me not always enough. So what you can do is you can modify it And I just cut and paste it to make my life a little easier To make it so that I can expand that range And make that a bigger number for lack of a better term. So now if I go in here You'll see that I actually have a bigger range in theory Because it's going I think more to the left right so so if we look at the man page It's all right here and Basically, it's like it's like the start and then account right so What it's yeah, I guess now in retrospect the example is not great, but so it's like I start here at whatever this is 100,000 and go 65,000 after that right whereas this one it starts at 10,000 and go 65,000 after that So it's basically just kind of making a big range of UIDs that I can play with you can do the exact same thing with the G UIDs And I'll just show that Which I think the only syntactic different is that you say G here And then we can see that I have these G IDs. Yay exactly the same But the reason I wanted to kind of give all this context or whatever is because you will see Potentially see different and slightly subtle errors Around the files you edit or around the containers you try to use or whatever that are all related to this problem where as your individual user so my you know my Kind of L white user here Is just user ID 1,000 and G ID 1,000 when I'm trying to run a container I'm running all those processes right just like I would be as if they were running locally But I'm trying to write create this kind of fictitious Computer which has a whole mess of other user IDs. So I need to make sure I can give them user IDs And you also can see this cool command out of pod man which I don't think I even used but it'll actually show you the users Mapped to their host user and host group. So as you can see I have this one So I was running a container in the background just basically for this example called based on Grafana And as you can see there is a user inside the Grafana container called Grafana It is of user group root inside the container. So in theory if I do exact IT C0 f bid bash And then do ps you can see actually ps minus a f maybe I don't know how to get group out of PID but the point is There's got to be a way right? But so as you can see it's running as user Grafana if I look at user share Grafana, I bet it will be As root and even if I do this So it's of group root and if you notice it has read only rights for group for anybody in the same group Long story short you can actually see that mapping of The users inside the container to the users and groups outside the container this Host user is one of those subordinates this host group is one of those In this case, it's the same group But you kind of see how it works so like I said I Wanted to give a lot of context because this is one of those scenarios where You run into some weird errors And they all are basically Kind of from this problem This not that That is a really quiet all of a sudden by the way. Oh Mike my voice. Yes, like you're further away from your mic Is that better you go now? It's better So the most common one that I've seen and I even think we saw that I think what led to this is that we saw this on the show was this one here when you try to do a pod man poll and it's using an ID inside the container that is a privileged user and Sometimes when you run it you'll get a similar one, but you'll get a couple of others too So just if you see something about the IDs It's related to The fact that you're trying to use these IDs and it's kind of a non normal way And so you're you need these subordinate user IDs and group IDs So let's wrap it up there. Um, let's talk about next week real quick. Yeah, so next week. I'm out I'm having my neck eotomy on my left side of my neck. We'll just call it that You'll be joining a frat. No Jesus the The show will be host will still be here though, right Langdon and Andrew are Playing second fiddle to each other. I guess Maybe I don't know like so Andrew who hosts the show that comes up next on the channel at 11 a.m. Eastern Ask an Open Shift admin will be hosting this show and Langdon will be stepping in to host Andrew's show So it'll be a Langdon and Andrew morning next exactly. Yeah But I will everyone's cigarette, right? I'll be here on Tuesday. I'll be here on Thursday, but Wednesday. I'm done I'm down for a count. I'm like I see myself constantly kind of thing. I should figure out how to do an ice bath I'm just my neck. I Think they call that a drowning Anyways, so yeah, that's what to expect next week and We'll worry about the week after that Well, so so next week we do have a guest so Yes, so we'll have Brian cook will be here to talk about the container health index And Andrew will be here to help me talk to him And then I will be joining his show. I'm not sure what he's covering that day But it's mostly it's a it's more of a question-based show so I think it's kind of a little bit what comes up from the audience as well as You know kind of what comes up in his day-to-day life between now and then so we will Be talking about all the things there the following week obviously or maybe not obviously is summit Part due as I saw somebody refer to it or summit the second second phase Which we've talked about on the show before but it's kind of more of the general sessions kind of components So a little bit deeper dive a little bit more things like roadmaps and stuff like that may be coming up About the various individual things that we do at Red Hat So you should definitely go check that out. It's free, you know, if you if you can't find it I'm pretty sure it's redhat.com slash summit I'd drop the link to it in chat. Oh, okay. So yeah, so summit part two is coming up We will be dark that week But then we'll be back the following week to give you a little recap of what we saw and thought was interesting And you know, hopefully you can share with us what you saw and thought was interesting But yeah, so Should be like I'm really looking forward to the container health index one next week I've been trying to get that scheduled for quite some time and it's something that I think doesn't get anywhere near enough publicity about, you know, which is something we do and offer around our content and From a little bit of conversation with Brian previous to this Hopefully we'll also be talking a little bit about some other places that it might land or where that experience can can kind of be enhanced And that kind of stuff, which I think will be interesting Cool But I think is that all we wanted to cover. Yeah, I think so. I think so. Yeah Yeah, it's always ask us questions. Let us know in discord. Yeah, you know, free to ping us in discord any time I actually have it set up so it will ping me During off hours that doesn't mean I'll respond But if you have an urgent question So, yeah, thank you very much for tuning in today, thank you Langdon for our journey through the illustrious bits and pieces Yeah, I don't think I ever realized how Complicated it is and then how like like you really are doing this kind of weird anomalous thing and it contains Yeah, and so, you know, I think it's it's important to kind of understand what you're doing So then you can kind of extrapolate the solutions when you run into some of the problems that you run into Exactly. So thank you all for tuning in. As I said, join us for asking up shift admin here at 11 a.m. Eastern time We will be talking about alert manager configuration and customization. So all you people with Kubernetes clusters that want to get alerts when things go bump in the night Please tune in and we will talk about Prometheus and all the fun things that it brings to the table Until then stay safe out there folks