 Welcome. So today we're talking about inodes. Also, we'll leave a bit of time because there's TA evaluations the department wants. So we'll leave a bit of time for that as well at the end. So let's get talking about inodes, which we left off talking about, you know, indexed allocation and how that could be pretty good. But our file sizes we could support are only something like 16 megabytes, which isn't that great. So what an inode would do and this is the predominant data structure in the kernel when it comes to file systems. That's where all the information about the contents of a file actually lives. So that's where the mode is stored for a file and the mode is all the permissions. And then there's space for the owners, which is a user ID for the user that that belongs to and then the group that that belongs to. And then there's some timestamps to keep track of when a file is created, accessed and modified last, which you could of course change it will. Then there is a field for the number of blocks that this file occupies. And then there are a bunch of pointers to point to those blocks that the file occupies. So there is a bunch of pointers. There is some pointers that point directly to blocks, in which case you should use them directly. And that's as an optimization for smaller files. So if a file only takes up a few blocks, well, it doesn't need to use a block for all the indexes. It can just point directly to the blocks that it occupies. And then there is a field for single indirect, which was an idea for index allocation before. So that's to represent, you know, big er files in the smaller ones, but still not that big. So single indirect block points to a disk on the block or a block on the disk. And then that block contains all of the index nodes. So it just contains a bunch of pointers. And then similar to how we have multi-level page tables, well, the same idea applies to files. So there is one pointer that should be treated as a double indirect block. So that points to a block that are full of pointers that point to another block that are full pointers that then you can that then point to blocks. And then finally, to support even bigger massive files, there are triple indirect blocks, which is like a three-level page table. So it points to a block, which points to a block, which points to a block, and then all those have the index bits that actually point to the actual blocks on the disk. So any questions about that? It's the same idea as page files except now, instead of just having, you know, on our page system, we just had three levels and that was it. Here we have a, we have a varying number of levels, depending on how big the file is. So if it's little files, we don't have to worry about that. Then for slightly bigger files, it just points to a single block, which is full pointers, and then we have the double and then triple indirect ones for larger files. So the goal of this is to be efficient for small files and also support large ones. So the inodes will hold all the metadata and pointers to blocks, and we'll see how this comes into play when we look at files. So if you're a smaller file, you don't have to consume space for pointers, they're just stored directly on the block, but larger files would have to follow the blocks depending on which one they use and how big they are. So there's also a small optimization here where if it's a very small file, it can actually hold its contents directly on the inode itself. So here there's like a number of direct blocks, single direct, double indirect, and triple indirect. Typically you will have 12 direct blocks and then one single indirect, one double indirect, and one triple indirect. So you'll have 15 pointers. And if they're each four bytes, then you have 60 bytes of information. So one of the optimizations is if your file contents are less than 60 bytes, I don't need to point to any blocks, I can just use that information and store the contents of the file directly on the inode itself without having to waste any more blocks. So given that, now we can figure out how big of a file we could support. So this is the scenario I was talking about before for an inode. So it could store 12 indirect blocks, one single indirect, one double, and then one triple. So given that, and then each block is eight kilobytes in size, a pointer to a block is four bytes, and indirect blocks only consist of pointers only. So it's however many pointers you can fit on a block. So now we can answer the question, well, under this scheme, what's the maximum size of a file indexed by that can be represented by this inode? Well, it's the same thing as last time. We still have the same number of pointers that we can fit inside of a block. So that is two to the 13 divided by two to the two. So we can fit two to the 11 pointers on a block. Then if we want to calculate the total number of addressable blocks, well, in this case, we have 12 blocks that can be directly addressed just from the inode itself because those are the direct ones. And then for a single indirect block, it would point to a table and that single table could hold two to the 11 because that's the number of pointers. And then if we had a double indirect block, we have to take two to the 11 to the power of two because each entry points to another table. So it'd be two to the 11 to the power of two. And then for triple indirect blocks, it would be two to the 11 to the power of three. So instead of adding all these numbers together and having a not nice power of two, we can just estimate it to be the largest one since I'll probably dominate the size of all of them. So to the 11 to the power of three is going to be two to the 33. So the total number of bytes that we could hold in a file would be two to the 33 blocks times how big a block is. So a block is to the 13, which would mean the maximum file size we could have is two to the 64, which would be 64 terabytes for a single file. So that's more like a file system we would have nowadays and this is the actual structure the kernel uses. So your maximum file size, there is a maximum and it's 64 terabytes for a single file. It'd be slightly over this because we didn't estimate estimate. So you could have this many more blocks, but you know ballpark, it's about 64 terabytes. So there is a so how files get represented are the entries in your directory are not the actual files themselves. Every entry in a directory is just a pointer to an inode. So it just records the inode that represents whatever file that is. So your todo.txt, if you have it somewhere on your drive, it looks like it's in a folder. Well, if you know the structure of the file system and how Linux works, todo.txt is actually going to reference an inode, which will be some particular number. So in this case, assume todo.txt points to inode one, that would contain all of the contents of the file. So that would contain the metadata, what blocks it points to all permissions and things like that. And within a directory, all it would record is the name and what I know it points to. So the directory, they don't a directory doesn't have that much in it. All it is is pairs of name to inode and that's it. So hard link points to one inode and typically this is the case if you have a file that you just create there, it points to one inode and nothing else points to that inode. But what you can do is create something called a hard link. So multiple hard links can point to the same inode if you know what to do. So todo.txt and b.txt can point to the same inode and when you remove that file, it only removes the hard link and that doesn't mean that content of the file is actually deleted because something else could be pointing to that pointing to that inode. So POSIX only has an unlink call and that's what unlink does. All unlink does is remove this and like if you did RM, b.txt while there's no actual remove system call, what's going to happen is they're going to be unlink and unlink would just remove b.txt. So now b.txt is no longer pointing at inode one but since todo.txt is pointing at inode one, well it's not going to actually be deleted. So then there's something called a soft link. So a soft link, all that does is point to the name of another file. It doesn't point to an inode, it just points to a name. So when resolving whatever file that is, the file system is redirected somewhere else and it reads that content. So if I have create b.txt as a soft link, instead of pointing to an inode, it just points to a name. So it could point to something like todo.txt and if you want to read the contents of b.txt, well you have to know from b.txt you have to follow it to todo.txt and then you can get the inode and then actually figure out the contents of the file. So we can see when we play with this too that sometimes soft links, the targets can be deleted so we could delete todo.txt and then you can't resolve the target of a soft link, although it still exists and that would be a condition you have to cover in your programs that hey, the files I accessed might actually be soft links and they might not actually point to somewhere. So we're soft links? Yeah, soft links are essentially like shortcuts in Windows. So they just point to a name, point to a path. So in Windows if you have a shortcut and then you delete whatever that's a shortcut to, then your shortcut doesn't work anymore. So same idea as here, it's just the unixy way of doing something. So let's take this example so we can play around with this and see what would happen. So in this say we have, we touch todo.txt and all touch does if you haven't seen it before is it just creates a new empty file. So if we look at our directory, we just have one file in it called todo.txt and now we can actually expand our skills a little bit. So we know ls-l to show kind of more columns. So over on the left here, there are all the permission bits and then here is a number of links which we didn't know before and then that is the user that owns the file, the group that owns the file and then the date modified and all that stuff. So if we add in a i, well we get a new column here and that new column all the way to the left is the actual inode number that todo.txt points to. So what we can do is create a hard link. So has anyone used ln before like dash s? No? Okay, so ln-s is how you make a soft link but we'll see that in a bit. If you just use ln that creates a hard link. So if I do ln, the first argument is a hard link to what? So this is a hard link to whatever todo.txt, whatever inode todo.txt is and I'm going to call the new entry into the directory b.txt. So if I create this, whoops I hit a bad key, if I create this then I have a file called b.txt and now if I look at it through this I can see that the number on the left column is the same so they have the same inode. So if I go ahead and edit todo.txt and say hello, so I save it, now if I cat it what should I see? Hello hopefully. So I can cat it and say hello and but now what would happen if I cat it b.txt? What should I see? Any guesses? Hello? Oops. Yeah, so if I cat b.txt I see hello because they're actually pointing to the same inode and the inode points to the contents of the file. So I can access the same inode through todo.txt or b.txt. It doesn't matter which, right? I could edit b.txt, edit in from b. So if I go ahead and edit b.txt and then just look at todo.txt, well I can see that it's actually the same contents of a file because it's the same inode so I can edit it directly. So the other thing we could do is create a soft link so it's the same, works the same way as creating a hard link except you just use dash s and that means it's a soft link. So I'll create a soft link to the name todo.txt and I'll call this c.txt. So if I go ahead and look at the directory again, well I'll see that there's this new entry called c.txt. I should move myself out of the way. So I see this new entry called c.txt and c.txt, whoops, here I can increase it. So c.txt you can see it just points directly to todo.txt but all the way on the left here it's actually a new inode. But because it's pointing to todo.txt if I cat c.txt well it will just magically go ahead and resolve that. So it knows that c.txt is a sim link so it would go ahead and access todo.txt instead and then get the inode from todo.txt which would then lead us directly to the contents of a file so we can display it. So the other thing we can do is we could, now if we move todo.txt to something like d.txt well all that does is change the name of todo.txt, it doesn't change anything else. So if I do an ls again I can see the d.txt well it was todo.txt move just simply renamed it, it didn't actually move anything unlike its name. So its inode is exactly the same as it was before I just changed its name. And now we can see in red that I broke the soft link. So the soft link isn't guaranteed to exist or point to anywhere that you care about it's just a name. So c.txt just has an entry that says hey I point to todo.txt and now I don't have anything named todo.txt anymore. So if I try and cat c.txt well I'm going to get no such file or directory which is a really confusing error because c.txt exists so that's not my problem so you might, if you don't know what a soft link is you might get that error and be like well that's stupid c.txt obviously exists but what it's trying to tell you is the thing that c.txt points to doesn't exist which is a bit of a pain. So and then finally we can go ahead and you know remove b.txt which of course removing b.txt doesn't change just removes that entry that points to that inode but I still have something else that points to that inode that d.txt so my original contents that I edited before are all still there even though I moved around to do.txt and changed up and deleted one thing which didn't actually delete anything it just removes entries in the directory. So any questions about that fun stuff? Cool so if you were to write out what the file system looks like before and after the remove commands well it would look something like this so before the move we created todo.txt which would have created its own inode and then we created a hard link where b.txt points to the same thing as todo.txt so they would both point to the same inode and then c.txt is a soft link to todo.txt so hard links point to inodes soft links point to other names so before the move this is what my file system looks like and then after the move all I'm doing is renaming todo.txt as d.txt so it's pointing to the same inode I just changed its name but now c.txt is pointing to a name that no longer exists anymore so you could put it in black to say that hey it doesn't exist anymore and then if I remove b.txt all that happens is I just remove that name in a directory so the name b.txt no longer points to that inode so in Unix everything is a file so directories are just files of type directory there's additional things so files can be a regular file which is what you're used to a block device so you which means you can only access it in big blocks so that's represents like a hard drive or an ssd something like that we've seen pipes and we've seen sockets before so just so we know directory inodes do not store pointers to data blocks all it stores are file names and pointers to inodes so it's just a bunch of tuples that are just file name inode what are all what are they all so looking at that too let's go back to our example here let's recreate that hardlink so I'll create a hardlink to d.txt called b.txt again so if I do that two things point to the same inode and then if you want to read the contents of an inode well there's a stat system call and also a stat command you can run so the stat command will tell you a bunch of information about what you just accessed and then what is specifically on the inode so this is the file to do.txt or b.txt which that's gotten from the directory and then the rest of the information is the inode itself so it is 22 bytes large it occupies eight blocks my io blocks are four kilobytes it's on some device so and then it's inode number and then we can it actually keeps track of the number of links there are to it so right now there's two links there's b.txt and d.txt so it keeps track of that information on the inode itself so the kernel actually knows when it can delete the inode so the kernel can only delete the inode if there are zero things pointing to it as long as there's one thing pointing to it it can't actually delete it and when you are on it you don't actually know if it's deleted unless you know ahead of time you check that there is only one link to it before you deleted it so then you can see the access the user id the group id and then all the access modify change timestamps and there's also a birth timestamp as when i created it so if i go ahead and say i remove d.txt and i stat b.txt again well now i can see that my number of links decreased from two to one so i'm one step closer to being able to actually delete it if i'm the kernel so any questions about that so after that we should have a good understanding of what's kind of stored on the inode itself so the file name is not stored on the inode itself that is only inside the directory because the directory keeps track of names to inodes and inode itself has no idea what its name is it doesn't really care so also to that it doesn't know where it is contained because it could be contained in multiple places at once it would know the file size it would know the file type and inode would also not know the number of soft links to a file because you'd have to follow a bunch of links from the directory and it just doesn't care so consequently it would not know the location of the soft links it would know the number of hard links to the file so we saw that in stat so it knows how many hard links are to it and the kernel has to maintain that number to make sure it is correct but it's not going to know the location of the hard links so knowing the inode you have no idea of what is actually pointing to you without actually traversing the whole file system so there's no way to go backwards from inode to a bunch of names but other things it would have on it that we saw is it has the access rights or permissions it has the time stamps it doesn't store the file contents it points to blocks of the file although sometimes if it is optimized it can sometimes have the file contents if it is actually small enough and then lastly it would definitely have an order list of data blocks because that is the whole reason we have this inode structure in the first place so here's the answers to all of that so again c is because the file contents if they're small enough will sometimes fit on the inode itself so I can go ahead and verify that and show you so usually the number of blocks that are associated with the file it will tell you if you stat it so let's see in here in here I see a c.txt which points to to do.txt but I can stat c.txt and see the actual contents of the inode itself because it's its own inode so if I look at that I can see it knows it's a symbolic link here but curiously it is pointing at zero blocks so it's not actually consuming any space on my hard drive so and the size of the file is only eight because it's not because it just holds the content of what it points to so its size is eight because to do.txt is exactly eight bytes long so all it holds so that shows that it only holds the contents of sorry it only holds the name of what it points to and that is the only thing it contains so what the optimization here is because size is eight well usually usually in the inode itself well these are all four byte pointers and then we're going to have something like 12 direct one indirect one double indirect and then one triple indirect so if we add all that this means we have 15 pointers which would also equal to 60 bytes since each pointer is four bytes so the optimization is if this size if the size of the file is 60 bytes or less it's not going to store anything it's not going to store any pointers because it can actually fit in this 60 byte so I reuse the space that I would have for pointers and I actually put the contents of the file there so in this case if the symbolic link well the only thing it contains is the name to do.txt so the first byte would be the character t then o then d t o dot t x t so I would reuse the space for the pointers and just put in the string that the sim link points to and that is the whole thing so that's a good optimization otherwise the file would have to take up at least one block and if my file is only eight bytes long and my block is eight kilobytes I'm essentially wasting eight kilobytes every time I make a sim link and usually your sim links the names you're going to refer to are going to be less than 60 bytes so this is a worthwhile optimization so any questions about that okay so that's pretty much it for my notes so the file system will again like we talked about before use a cache to speed up writing to disk so we know that writing to a disk is slow and like always we'll use a cache to speed it up so what you can do is cache file blocks and cache them in main memory and that would be something called a file system cache and then you would create the cache because well you know that if you reference a block it's likely to be referenced again if that file represents something like a database or something like that you'd constantly be revising entries and modifying them so that would also be an example like we saw before of temporal locality and then logically blocks are near it are likely to be referenced next so things could be spatially located so if I read a file sequentially well I know that I'll be reading the next contents of a file so one optimization you could do if you are the kernel is go ahead and preload that information into your cache so it's there just in case something uses it and then you could also use that file system cache to record writes to so you could record all the writes to memory and not write them out to disk until as late as possible and that is why there are special system calls to actually trigger writes so there is a flush and a sync system call and that actually triggers that file system cache to actually be flushed to disk so you know for sure that you're actually going to have that information written to the disk so if you lose power or something like that then you actually don't lose data so one of the last things about file systems are something called journaling file systems which are file systems that try and ensure some integrity so now we know all the steps you have to do if you want to delete a file on unix so deleting a file assuming that's the last thing last reference the inode involves three steps so I remove its entry from the directory and then I would decrease the count the number of counts to that inode and if it's zero then I would release that inode to the pool of free inodes and then as the third step I would whatever blocks that inode was pointing to I have to set those all to free now because other files could now use that because I'm not no longer using it so I should return it to the pool of free disk blocks so if you actually care about data integrity well a crash could occur between any of these steps so say we crash between step one and two well that inode would still look like it is in use and all the blocks it would point to are still occupied that inode also is still occupied even though nothing actually refers it anymore so we would lose some information there that we'd never be able to get back and similarly if it crashed between steps two and three well that inode would would be released so it could get reused but now all the blocks it points to are going to still appear to the kernel as being occupied even though that they're free and since we can reuse the inode now whenever you power the machine back on well all those blocks are going to be lost till the end of time so that would give you a storage leak and what a journal does it basically just contains all the operations in progress so if a crash occurs you can recover so the journal would write an entry before it starts those three steps that says hey i'm going to remove this entry in the directory and be like okay i'm starting it now and then go through the steps and only after it's done it would remove it from the journal so if it crashes it would still be in the journal and then you can retrace your steps and then knowing how you do the steps you can see at what step something broke down so that's the point of a journal so if it powers off anytime between there you can recover because you know what operation crashed so you can go ahead and actually release the blocks or something like that if that's the step it crashed on so inodes were the hybrid allocation strategies so they offer way greater flexibility over contiguous linked fat or indexed it's basically indexed but we're taking an idea from our page tables and everything is a file on unix names in a directory can either be hard links or soft links hard links point to an inode and soft links point to a name so we'll take the rest of the time to do the uh to do the ta stuff and uh just remember i'm pulling for you we're all in this together