 Welcome back to 353. So today we get to talk about index, well, iNodes, short for index nodes. So where we left off, we could only support like a 16 megabyte file, something like that, kind of lame. So we will support larger files and also today it will be a fun one because we'll figure out exactly what LS shows us because I'm guessing not everyone knows all what the numbers mean. So first off to talk about an iNode. So this is how the kernel represents a file in your file system. So iNode is basically just a big C stretcher with a whole bunch of things on it. So one thing, it needs to point to the contents of the file but it also needs to source some metadata about the file. So some things an iNode would store is like the mode. So like, can you read it? Can you write it? Can you execute it? The owners, so there's two. There's like the user ID that owns the file and then the group ID that owns the file. Then there's some timestamps. Like when was this file created? When was it last modified? Fun things like that. Then there is a size block count. So how many blocks in your file system does this file actually occupy? And then instead of just like what we had before is we pointed to a block that was full of pointers. Well, it turns out that would be pretty wasteful if your file only needs like one block to store all of its data. So you would need like a block to store pointers and then that would point to exactly one block. Seems kind of wasteful. So how an iNode works is it aims to be really efficient for small files and also support larger files. So there is a kind of variable amount of indirection we can do. So typically the iNodes will explore the current Linux ones, have 15 pointers. 12 of them are direct, like point directly to a block. So if the file is 12 blocks or less, all of the pointers are stored on the iNode itself. It doesn't need to consume an additional block on your disk or file system or anything like that. And then for larger files, what we talked about last lecture would be considered a single indirect. So it would point to a block that is full of pointers and then all of those pointers point to data. And then to support larger files, well, essentially you use the same idea from page tables. So there would be a double indirect block. So that points to a block full of pointers. And then if you follow that, that goes to another block full of pointers and then that points to the data. And then finally, there is the triple indirect block that would just do the same thing three times. So it points to a block full of pointers, which points to a block full of pointers, which points to a block full of pointers. And then that actually tells you where the contents of your file are. So they aim to be efficient for small files and support larger ones as well. So basically an iNode just holds metadata and pointers to blocks. If you are a very small file, so if it's only like a single line, well, then it's only going to use the direct pointers. It's not going to have to use a block on the file system to actually store the pointers. They'd just be stored on the iNode itself. And if you have a larger files, well, then you just have additional index nodes with pointers that point to additional blocks. And very small files, like one optimization you can do for some types of files is that, well, assuming we have like 15 pointers and they're four bytes each, that's 60 bytes, if the contents of my file is less than 60 bytes, hey, I can just store the content of the file directly inside that space since I don't need to point to anything. So now with an iNode, so the typical Linux iNode has 12 direct pointers one single direct pointer, one double and then one triple. And then, well, if we have the same scenario as before where our disc blocks are eight kilobytes and a pointer is four bytes and we'll assume that the indirect blocks just contain pointers, nothing else. Well, then we can see what the maximum size of the file managed by this iNode actually is. So, same thing as before, where we have to figure out, okay, well, how many pointers can we fit on a single block? So that would be the block size divided by the size of the pointer, that gives us two to the 11 like we had before. But now the number of addressable blocks, well, that's how many blocks I can point to. So through the direct blocks, I can point directly to 12 blocks. And then through the single indirect block, that's just two to the 11 because that's how many can fit on a single block. Through a double indirect block, it's two to the 11 to the power of two. And then through a triple, it is two to the 11 to the power of three in order to not, we don't have to be super exact because this term is a hell of a lot bigger than all of the other terms. So if we're just talking about the maximum size of the file, it would be approximately two to the 11 all to the power of three, which is two to the 33. So the total number of bytes we can have for an individual file would be two to the 33. So that's how many blocks we can point to only through triple indirect. And then times the size of a block, which was two to the 13. And then the maximum size of a file we can support is two to the 46, which is 64 terabytes. So probably a big enough file, right? Probably pretty good. So let's see, what if I wanted to support a file larger than 64 terabytes, what should I do? Or I could pose it the same virtual memory way. If I wanted to support, I don't know, a larger virtual address, what do I do? Yeah, yeah, add more indirection so I could make like a quadruple indirect block, right? Just stack on the layers. If I made a quadruple indirect block, well, that would get us up to what? Like 128 exabytes, which is probably a pretty big file. So should keep us all right for now, but 64 terabytes, probably pretty good. All right, so a questions about that calculation all before we get into the fun stuff. All right, so fun stuff. So in your file system, you may not have known this before, but there are things called hard links. And this is all files actually are. So file names are just for humans. So a hard link is basically a name that points to an inode. So any entry inside of a directory, AKA that's what a file name is, is called a hard link and it points to a single inode. So we can actually play around with stuff today. So if I create a file, let's say a.txt and say, I don't know, this is a. So if I do ls and look at it, anyone know what all these numbers mean and what the hell ls is trying to tell us here? So if someone wanna tell me what this means, yeah, permission, so I can, so the owner of this file can read and write it. Anyone else can read it and anyone in the group can read it and anyone else can also read it. What's this number mean? One file, could I ever make that number go up then? Is it the size? No, so this number is the size of 10. So it's like 10 bytes large. So close, does anyone have any idea what this number means? Yeah, the level of indirection, that's a good guess. So if I make the file much bigger, that number should go up. So two, yeah, so maybe it has to do, goes up when I make the file bigger. All right, so these two things, so that is the user that owns it, that's the group that owns it. What's this? Yeah, yeah, the time that I last modified it probably. So let's see if a minute's passed, yeah, probably a minute's passed. So if I go and change it, hello A, if I ls again, oh, the time got updated, so I just modified it, so it's the last time I modified it, and that's the name of the file. So we don't know what this number means, we have a guess, so let us go back real quick, and then we can play with something. So you can actually create multiple hard links to the same inode, and what I can do is I can create another hard link called like b.txt that points to the same inode, which would be like the same content, same metadata, same everything, and we can go ahead and try that out. So I can create a hard link by using the ln command, don't ask me why they named it that. So I take, as the first argument, I use this name to point to an inode, and that's the inode that I'm going to also make this new name point to, so I can create a new file called b.txt that points to the same inode as a.txt. So let's do that and run ls again. Oh, shit, that number went up, hmm. Any other guesses what that number might mean? Yeah, yeah, close, yeah, the number of files that points to the same inode, yeah, yeah, the number of hard links to that inode. So another mystery we can get, so with ls, I can do dash l and then there's a new one, so I could do a little, like lowercase i, that'll also give me a new column that shows me the inodes. So if I do that, the first column here is the inode and it's basically like a big array, so you just get some numbers. So a.txt points to inode 5,314, b.txt points to inode 5,300 or 314 as well. So this number here is just the number of hard links to that inode. So I created a hard link to it and if I try to look at the contents of b, well it says hello a, if I try and look at the contents of a, it says hello a because while the inode contains where on the disk, what blocks, where the content is, so since they're the exact same, they'll read the same thing. So let's see, if I modify b.txt, oops, so if I modify b.txt and I look at the contents of the file, it says hello b, what should I see if I look at the contents of a.txt? We'll still say hello a or we'll say hello b. Any a's in chat? Wait, oh, it's gonna say hello b, yeah. So if I cat a.txt, I get hello b because well I changed like all a name is, a file name is, it's just what inode that is and if I modify the contents of the file, well that would modify the inode. So the inode would have to point to a new data block that has the new contents of the file. Since it's the same inode for both names, whenever I modify something, it's gonna be the same in both of them, right? And well this number, like when we LS, this number was pro, there's probably always one for all your files, so they kind of all look independent, but if you wanna screw with someone like a first year course or something like that, just start hardlinking like they're C files together and then they'll modify one file, it'll modify other ones and it will be great. Other questions, so if I change permissions for one, does it change for both? So let's change the permissions on a.txt and now if I look at it, I change the permissions for both because the permissions are stored on the inode itself, so if I change the permissions, it changes the permissions on the inode. The name is just a way for us lowly humans to get to the inode itself, so we don't have to kind of like the internet. If you had to like type IPs to go to Google.com, probably be kind of a pain. Same with files, so we just access them by name instead of just the number all the time. All right, any questions about that? Okay, well, how about a fun one? So since we know what this number is, let's see if we can figure out. So remember dot is the current directory and dot dot is the parent directory. So if we look at this right now, so dot, which is the current directory is currently inode 4295 and it has four entries pointing to it. Hmm, weird, although that one's legitimately weird. All right, so what are some of the names pointing to it because realistically it should be two but I guess some weird things are happening because I virtualized it. So of these four names to this, four hard links to the same inode here, what are the names of some of them? So one of them would be dot in this directory, right? And if I look at my path here, the current directory I'm in is called test. So where would another one be? So I'm in the test folder but probably in the one above it, right? So yeah, if I go backwards one and I look at it, well, Ist is given the name test and in test it has that inode, right? So an inode also stores the contents of what is in a directory. So one of the links to that inode would be the name test in whatever this directory is, this is 28 inodes. And then if I go into that directory, another name for it, whoops, another name for it is just dot in this directory. And then the other two, some weird container things going on here should only be two. And this directory is basically saying how many things are pointing to it as the parent. So I'm pointing to it and any time I make a new directory, so if I make directory, I don't know, test two, and I run it again, well, I just increased the number of references to this directory because within test two, it would contain like a dot dot and it would point back to me, right? So each time I make a new directory, inside of this it would increase by one because inside there it would have an entry to dot dot which points back to me. So if I make test three, same idea, I went up from five to six now and if I go ahead and I delete them, test one, test three, go back, look at, I'm back down to four. So yeah, you can think of like just names as just pointers, it's kind of a shallow copy if you wanna think of it like that, but realistically all the things in your directory, they're just names to inodes. So, any questions before that before we make it even weirder? Yeah, so the question is, is it possible to remove the hard link for dot? So like if I just, we can try, I assume this will be an error, but it says yeah, I cannot remove it, it's a directory. Okay, what if I just say, actually this will probably just nuke the entire directory. Yeah, let's do it. Okay, refusing to remove dot or dot dot. So, they're reserved, they have to be there, the kernel enforces that, doesn't let you delete them. All right, anything else? All right, let's make it weird. So, if there's a hard link, there's probably a soft link. So, we can create soft links, oh yeah, and just to note, so with hard links, hard links form a directed acyclic graph aside from like the loop for dot. Aside from that, your file system is like hierarchical, it's a directed acyclic graph, but if we introduce soft links, soft links are just like, instead of name to inode, one way to think about soft links is it's just a name to another name. So, in order to create a soft link, I just do ln and then do dash s, so that will create a soft link. And then, let's see, I can create a soft link to a.txt and call it c.txt. So now, if I look at it now, here, let's do everything. So now, I create, technically it needs a new inode itself to store the contents and the contents of a symbolic link is just the name it's pointing to. So, I created a file called c.txt that points to a.txt. So now, whenever I try to access c.txt, what's gonna happen is it's gonna be like, okay, well, I actually need to look at a.txt to find the inode itself and if I cat it, I get hello b. All right, any cool questions about that? Yeah, oh yeah, that's a good question. So why is the size of our c.txt five? So for a symbolic link, the size of it is basically the name that's referring to. So a.txt is five. So a, period, txt, five characters in total because that's all a symbolic link of storing is this name here. Yeah, yeah, so what's gonna happen if I do like an open or something on c.txt? Well, your kernel is gonna be like, okay, well, that's not a useful inode. I have to open it, read its contents, and it's like, oh, I need to follow a.txt, and then thankfully in this case, a.txt is a hard link, so then it would just get that inode. And yeah, and then just to show that RM doesn't actually delete anything. So if I do like RM, let's say RM a.txt, all that does is remove that a.txt entry. It did not get rid of that inode, so we decreased the number of names to that inode from two to one, and now there is just b.txt that points to it. And spoiler alert, this is basically what your recycling bin does. So in order to actually delete this file, if I have no references to it, so if I did RM on b.txt, we would essentially reduce this from a one to a zero, and then we could actually delete the file and actually get rid of it. But how your recycling bin basically works is the recycling bin will just create a hard link to that inode so that there's still at least one reference sitting around and your operating system will not delete it. And that way, if you want to recover the file, well, it's pretty simple. You go into your recycling bin and just say restore file. That would, it would have to remember what the name is you deleted it at, but all it would do is just use the same name and then point to that same inode that was pointing for and boom, your file is restored. And that's also why if you want to restore a file that's like 20 gigabytes, it takes no time at all because all it has to do is just make the name point to that same inode that still has a reference up. All right, cool. All right, any other questions with that? Yeah. Oh yeah, that's a good question. Can I have a soft link, point to another soft link? Yeah, so right now, so the other question is why is the permissions for this one like weird? Why is it in this color? Well, it's in this color because if we do c.txt, that points to a.txt, if LS checks if a.txt actually exists and right now I removed it, so there's nothing called a.txt. So we'll get some confusing messages. So if I try and do cat c.txt, c.txt clearly exists, a.txt does not exist, but if I try and cat it, it says c.txt, no such file directory, which is a bit misleading because c.txt exists is just a.txt does not exist, but you get the same error message that says no such file or directory. So what is your question? Can I make a sim link, point to another sim link? So let's even make this more fun. So let's create a sim link that points to c.txt and call it a.txt. So it doesn't really like that terribly much. So that kind of looks like what? An infinite loop, right? So c.txt is a.txt, then a.txt is c.txt, da, da, da, da, da. So it should go as an infinite loop. So what happens if I do this? Probably nothing good, might infinite loop, might give me an error message. Well, let's see. It says too many levels of symbolic links. So terminology wise, soft link, sim link, symbolic link all mean the same thing, just depends who you want to ask or what day it is, I guess. So here we catted c.txt and we got a nice little error that said too many levels of symbolic links. So there's a little bit of a mystery here of who was smart. So did whoever make cat, were they smart and they gave up after a while or was the kernel smart and just gave us an error code? So how would I figure out which program is smart? Like what's smart? Is cat smart or is my kernel smart? Yeah, could I make an obscenely large cycle? Yeah, I could, I just don't want to type that. So I could, it gives up after a little bit of time though. So here, how would I figure out what's smarter? Is the kernel smart or is cat smart? So what's one of my favorite tools to see what the result of system calls is? You remember what it's called? Yeah, strace, yeah, I can see, I can just strace it and see what was smart. So if I strace it and I look to do, so it writes. So we can see here, it only tries to open c.txt once and I get an error code that's just eloop. So my kernel is smart, my kernel is the one protecting me. So I opened it and I got an error code negative one, eloop. So too many levels of some block lengths so my kernel is the smart one here. Yeah, anything else we want to do with this? So yeah, so looks angry at me if I get rid of it. So the nice thing about symbolic links too is since it's just pointing to a name, I can get rid of the name and then create a new file that would get a new inode and then suddenly my symbolic link works now. So otherwise, like if they were pointing to the same, like if they were hard links with each other and I tried to make it point to something else, like I'd have to make sure that they're always the same and always point and like make sure that A and B always point to the same inode even if I remove the file, create a new one, kind of gets a pain. So for that, you generally use a symbolic link so you just point to the name and then that's it. If you change the inode, no problems whatsoever. All right, any other questions or any other weird things you want me to do with this? And yeah, another thing too, so like when I remove and I bring down its references, so if I actually do like RM of a.txt and now there's no more references to it, well, spoiler alert, your like zero ring every single data block you used for that is slow and typically your kernel will just like mark those blocks as now unused. So if you actually want, if you accidentally like delete a file, typically you can recover it because the kernel doesn't actually like zero out all the inodes or anything like that. So if you just kind of scan your file system for kind of inodes that aren't really referenced, they'll probably still be mostly intact and point to valid data blocks and you can actually recover your files if you know what inodes look like and that is also someone you pay like thousands of dollars for for data recovery when you accidentally delete a file. But that's basically how that works. All right, any other questions about that? So, oops. So yeah, we can go through a little example then. So, oh yeah, we haven't got, so this touch command. Yeah, I'll show you why I'm slightly distrustful too. So touch command by default will just create a new file if that file doesn't exist. What it's supposed to do is like update the timestamps of like when you last modified it. But if I do this, if I touch to do.txt, it would create a new file. So I would have a new entry in this directory called to do.txt that points to some new inode. If I do ln to do.txt b.txt, well b.txt would point to the same inode as to do.txt. And then if I create a soft link called c.txt that points to to do.txt and then do a move. So move doesn't move anything. So all it does is rename the, just rename the entry in the directory. So that's why also if I move a 20 gigabyte file, 100 gigabyte file, whatever, it doesn't actually physically move anything. It's instant. All it's doing is changing the name. The inode is staying the same. And that's where the contents of that file actually are. So my file system before the move command would look something like this. So c is a soft link. So the way you model it is name to name. So c.txt points to to do.txt and then to do.txt and b.txt are hard links that would point to the same inode, just one.txt. So if I move to do.txt to d.txt, what that will do is create a new name called d.txt that points to the same inode. And then just to show that this now no longer exists. So c.txt points to to do.txt which is currently not in the directory. So I just put a dark background on it. And then if I do rmb.txt, while that just gets rid of that b.txt entry, there's still something pointing to that inode. So questions, any questions about that? Cool, we kind of know what LS does. Oh, no, nope, that was just tired. Okay. So in Linux, everything is a file. So directories, same idea. They're inodes, they have a special type. So they're called directories. Additional types are there's a regular file, while sim links a different file type. There's a block device to model like SSDs and hard disks. You can actually name pipes if you really want. So you could give a file name to a pipe. Can do the same thing for a socket. And directory inodes, they don't store like pointers to data blocks, but really what they store is just tuples of file names and inodes. And that's all a directory is. It's just a whole bunch of tuples of names and inodes. So if you looked, oops. So if you looked at the con, like the inode for this, the contents of that directory would essentially be there'd be a name dot that corresponds with this inode, a name dot dot that corresponds with this inode, a name b.txt that corresponds to this inode, and name c.txt that corresponds to this inode. And that's all a directory is. It's just like name inode tuples and that is the contents of a directory. All right, so this is a little thing. So what is stored in an inode? Well, the file name's not stored in the inode itself. It's stored as part of the directory. A inode also doesn't know what directories contains it. It does know the size of the file. It does know the type of the file. It has no idea what the number of soft links to that file is or the location. But on the inode itself, it actually stores the number of hard links to this file. Otherwise it wouldn't be able to figure out, but it does not know the location of the hard links. So you'd have to walk through your entire file system, starting at root. And then other things it would store is access writes, timestamps, and then the file contents sometimes, most of the time it just points to blocks. That actually has the contents of the file and it also, by definition, has an ordered list of data blocks. So, other fun things. All right, so you wanna see something fun with the touch command? So the touch command, this is also why I am slightly distrustful and I probably shouldn't show you this, but what the hell I'll do it anyways. So that touch command can just modify, modify dates as much as you want. So like, how much do you trust that date? Knowing what we know now. It's just kind of stored on the I-note itself, right? It's just, it's not that special. So I modified it March 19th of this year at, what was that, 322? So because this doesn't have time zone set. So yeah, do we trust that? So I will show you why I had to put in the lab handout that I do not trust a single date you have on your computer because guess what? You can change it to whatever you want. So that touch command, you can just give it dash D and well, I can change it to whatever I want. So let's see. I was definitely not alive at this time, so don't make any jokes, but let's do 1969, April 1st, and I'll change B.TXT. So now, if I look at it, oh yeah, I totally did, I totally did this file in 1969 and you should definitely trust that because that makes total sense, right? Yeah? Huh? Oh, so it won't show the time if it's like, it only shows the time if it's like within 24 hours. That's what LS does. But it just stored the time at like 0, 0, 0. But I could have set a specific time if I wanted, but didn't have to in a sense. Like, oh no, wait, what was yesterday? So like, oh, I was like totally working on my assignment last night on the 18th, right? Whoops. So yeah, I did March 18th. Happened to be right at midnight and so like it's fine, don't worry about it. So I showed you this. Don't do it. But like other people might not know you can do this and like who trusted the file timestamps before? Yeah, most people. So doesn't matter, works the same way with Git. So I guess this is a Git repository. So if you want to, this date is just stored somewhere. So you could modify that if you want, which is why the only thing that matters in terms of this course is like when you push the, when you push your code onto the server because that is run by me and I trust its clock. So it knows when you push code. So other than that, timestamps do not matter. You can modify them to whatever you want because I did not make this file yesterday. I made it just now. Normal lecture, great power, great responsibility. Don't do this, but you can mess with your friends if you really want, but don't take advantage. All right, any other questions about that, aside from like I will never ever do this, but now I've made everyone slightly distrustful of timestamps, right? So fun. So, okay, so other things to talk about and wrap up file systems. So writing to disk is really, really slow. So file systems will like cache to speed up writing to disk. So what typically will happen is, well, I might read the contents of the file from disk into memory and I'll just use memory as a cache for disk. So if I modify that file, I won't write it back out to disk, I'll wait until all the modifications are done and then I'll write it back out to disk whenever it's done. So file blocks, they'll be loaded into main memory and your kernel will maintain a file system cache because well, likely if you reference a block or if I'm modifying a file, I'm probably going to modify it again. And if I'm modifying the first block of the file, I might be referencing or modifying the second block of the file. So that would be spatial locality. I modify things that are close together and then I'm more likely to modify something I've just modified. So what actually happens if I modify a file, it will just only modify it in memory and not write it out to disk and there'll probably be a kernel thread that's running as part of your kernel that would periodically like write changes to disk if your disk is otherwise not doing anything. And you can do that yourself. You can like force writes to happen to disk with a system call called flush or sync and that essentially forces writes to happen kind of related to print F and doing F flush. So it's a different cache, but it's the same idea. It forces writes to happen. So this is also anyone ever use like the safely eject USB drive button. Yeah, so anyone know what that does? Yes. So the safely eject USB drive I've had happened before where like I plug in a USB drive, I copy a large file to it. I know my USB drive is slow, but it just says it's done. Well, it's done because it's all cached. It hasn't actually been written to the USB device yet. So if I yank out the USB device, well, it's not going to be consistent. I haven't like my kernel hatchly hasn't written data to it. And if I do safely remove USB device that essentially does that sync system call. So it will just wait until it actually writes all the data out to that USB device so that when you unplug it, the data that you expect to be there is actually on your USB device instead of it being like halfway through writing to it and then it getting corrupted. So that's what safely remove USB device essentially does. So questions, any questions about that? Cool, cool other little thing, right? I've had that happen before where, you know, USB device, copy gigabytes, it's like it's done and then I do a sync or safely remove and then it takes, you know, like five minutes because it's actually really, really slow. My kernel just kind of lied to me. So other little thing is for file systems. Oh yeah, another thing with file systems. So there are multiple things you can do with file systems. I could do the same copy on write thing for data blocks if I really wanted to. So there are file systems that support it. You've gotten experience with it, but same idea as with virtual memory just applied to file systems instead. There are also things called journaling file systems. So what that does is it aims to be like more resilient and lets you recover if you just like pull out your power or something bad happens. So if you were to actually physically delete a file using Unix, it would involve three steps. So, and this is if I'm like actually trying to delete a file, I would have to remove its directory entry so I remove its name to inode tuple in the directory. And then, well, if it's last reference to that inode, I essentially have to free that inode so I have to update my file system and say, okay, I'm not using that anymore, it's free to get used again. And then, well, I have to go through that inode and any disk blocks it was using, I can now reuse because that file doesn't exist anymore. And if you think about crashes or the power game pulled out or something, it could get removed, like you could remove power between any of these steps and then you would have a very inconsistent file system. So if I just removed the directory entry and then power went out, while that inode is going to claim in the file system to still be used even though there's no name pointing to it and any blocks are also going to still be used as wasting space. So it gets inconsistent that way. If I remove the power between steps two and three, so I don't have a directory entry to it and that inode is free and it can be reused and but I haven't actually freed any blocks that inode was using before. So just a bunch of blocks on my disk are going to be just sitting there wasting space, I'm not even gonna be able to recover them. So that's basically what the journal does is it contains operations in progress. So it would essentially write to someplace and flush the right so it knows what it's about to do. So it would just write an entry that's like, I'm deleting this file and it would not remove that entry in the journal. It's essentially like a to-do list until it actually completes that operation. So if I crash between any of these steps and I haven't checked that off my to-do list, well, whenever you plug in power the next time and boot up your computer, it's going to check that to-do list or journal and be like, oh, okay, I was in the middle of deleting this file, I crashed halfway through it so I can just retrace my steps and figure out what's inconsistent about it and then I can go ahead and finish and clean it up correctly and then check it off my to-do list and then actually recover. Yeah, so any questions about that? All right, cool. There's some Discord questions. How do directory, so directory date modified works by just whenever you last made an entry to it and then another question, just curiosity. Talking about removing a file, a friend used to tell me RM-RF is faster than UI remove file. If you click to move the file to the trash thing, why? Not sure about that, but