 Okay, so let's talk about another fun topic. So tempfile support. There are many workloads, there are test cases that create a tempfile and then hardlink it into the namespace. You know, it's not required. It hasn't been around forever. This isn't something that was in original Linux, apparently. So what is a tempfile? As you guys know, it's an unnamed file that, I mean, you can create a silly rename weird file name or you could make it hidden or do whatever, but the file systems don't have to have a name for it. The path name only has a directory and there's not much path. It's just basically a create and an open. So it's very similar to create. It is create, but the only difference is that this file is gonna disappear when it's closed. So there are currently eight file systems that support tempfiles, primarily BTRFS, EXT4XFS, but actually nine, or maybe miscounted here, nine of them. There's a problem though, and this is like an obvious inherent problem if you think about a tempfile. The tempfile is supposed to be deleted when it's closed. There are two types of file systems, right? There's file systems that have kind of a two-step create that creates a file and then opens it and there's file systems that have a one-step create where create returns the file descriptor that it just opened. So for half the file systems, the act of splitting the creation of a tempfile into two steps where you create a tempfile, close it, and then reopen it, defeats the whole purpose of a tempfile. My intent when I create a tempfile is to create a file marked delete on close, right? We have in the protocol spec a delete on close, but the instant I close it, it's going to go away. So you're gonna create the tempfile and then open it and it's gonna be deleted by the time you get to open. So you can defer close, that's possible. We have directory, or we have handle leases and deferred close support in SMB, but it creates the small possibility of races and it's also slower. So you're doing a slightly more chatty thing than you should and there's a slight possibility of some race someday. So some file systems can work around this by not actually opening a tempfile when it's being created. They can use a path-based call that doesn't return a handle, but that doesn't work for other FS. Once the link count gets to zero, it goes away, but this is also suboptimal. So here's an example of creating a tempfile from one of the simple examples. Here's RAM FS, what they do. Pretty simple, right? Get inode and then d underscore tempfile. It's very simple and then they export the .tempfile. This is trivial amount of code. So what is the code path that goes on here, right? So when you, the inode op tempfile is called with your mode bits and it returns an entry on a directory on it. So you do this path open at, do tempfile, vfs tempfile, tempfile. So in the calling sequence, if this isn't a tempfile, then it's gonna call do opath and then vfs open. But in the tempfile path, it's gonna call vfs open after it's done this create tempfile. So lookup op on the other hand just simply calls atomic open. So we don't have that issue of create, close. So it's easier in the open create path than it is in this one. This is the only create path that has this problem that I see, it's one of the few. So if you look in do tempfile, it looks up the path, calls out to the vfs for tempfile and then simply opens it. I mean, this can be kind of merged together, right? To be, if the file system supports this as one operation, have the tempfile return the, you can see may open returns the, so have it return the vfs open, right? So the vfs open returns file, right? So basically have the file system do the tempfile and basically like vfs open, have it do the tempfile create and return the file. This is similar to atomic open. So add a create.directory iNode operation called atomic tempfile and then that makes it for any file system that chooses to do is this makes an atomic operation, speeds up things a little bit, a little bit clearer to read as well and it avoids the open create close race for file systems that for which create is a handle based operation and it doesn't require changes to anything else. So it's a fairly simple one to imagine because as you can see in do tempfile we're just splitting the open artificially from the create of the tempfile and what it means from my case is that it would, you create the file market, delete on close. If you do a hard link to it, obviously the newly created link won't be delete on close. And then we mark it as hidden and give it some random name that doesn't show up in the directory. Would it be better to go through the already existing atomic open just with a tempfile set? I looked at that and it's possible. That brings up a topic for next year's LSF which is if you look at the file this stuff is in this code, it's kind of ugly actually. There's a lot of great VFS code. I don't think that the way we handle open and create is the easiest to read. VFS open, path lookup at, do tempfile. If you look at some of these functions I'm talking about, path open at and the lookup open, they're kind of confusing. There is a much bigger problem I see and that is that there's lots of cases where we unnecessarily do stat, does the file exist and open or create when we could have done one operation. So there's lots of cases for a network or cluster where we're calling two operations. But I sort of feel like- So when you say stat, do you mean user space is doing stat open? No, I mean that the act of doing a create very frequently, the act of doing open or create will revalidate a whole bunch of dentaries. Now some of that's, sure, we have to do. But when you get to the final dentary, in a lot of cases there's no need actually to revalidate the final dentary because it's implied in the open call and we could have done the error path by saving one operation on the wire by not sending the query across the wire to see if- So in any case, the open call itself could have been just sent saving one round trip on the network. So this is combining the lookup. Yeah, lookup and open. Right, and the lookup would fail. Yeah, and there's cases where we do this and cases we don't. But what I was saying was that reading this code at one in the morning last night was confusing and I've seen it a hundred times, but some of you guys may know this code better. When I say this code, I mean the bigger part of your question. It's trivial to imagine the change I'm describing, right? Where tempfile and open of the tempfile is just a call out to a file system to do the same thing if it's available, if that function's available. But looking at the bigger picture of how to merge this a little bit more sanely into, where's that thing called? Into VFS open. Anyway, there's cleanup that can be done that has nothing to do with the tempfile story in these code paths. And I think it's worth looking at. I think it might be desirable to at least try whether we, if there are no weird issues popping up, obviously with this, there's a tendency for VFS. If we could do it for all file systems. So not just add an atomic tempfile but and have another tempfile operation or just have a single one and convert all file systems. Yeah, I think it makes sense. And like I said, we have some test coverage here. So it's not like unheard of. I don't want to overstate the importance of tempfiles. I don't think NFS has. For user space, they are indeed quite important. We used them a lot. I mean, I guess, you know, Beecher FSE-64 XFS covers 90 something percent of the local cases, right? So, you know, yeah. I don't know, I don't think Fuse did, but. I don't know whether it is possible to combine the tempfile inode operation with the file open operation in all cases. I think overlay FSE maybe wants to do, create the tempfile, copy some attributes and then effectively open it. A union mount was going to work like that but union mount got dropped. I mean, it's definitely worth experimenting with this. Right now, well, creating the tempfile overlay does use that, but you can always create a tempfile without opening it, right? Well, that's what we do at the moment, but if we force combination of when you want a tempfile, thou shalt only take the same time, is that a problem? So where you don't have the option of creating a tempfile and then opening it and doing something between? I mean, it's not required to have a file descriptor, so I don't know why we would not allow creating a tempfile. I mean, that should be fine without opening a file. I mean, both, we can do both. I mean, from user space, this is a flag on, you know, creating a file, right? I mean, it's, I don't know. Just from the user space perspective, it turns a file handle, right? But anyway, I also have one more short topic if we have a couple, we have a few minutes left, right? So, you know, any feedback on this would be useful. We talked a little bit about these yesterday, and I just wanted to make sure that we're on the same page. So, you know, StatX is awesome. We can set and, you know, encrypt it at rest and compress the rest and some other things. If you look at the current StatX flags, these are the current ones, and did I miss any, David? I mean, it's a little bit tricky. I think I saw all of them. Some of these are actually not, it's kind of an interesting story here, but that's the current list, but it's also a little bit misleading because, and I'm not sure that, I don't know if this was intentional or not, but three of these flags aren't file system flags, they're VFS flags, so they're not persisted, right? So, compressed, for example, stored by D2FS, CF-C64 and TFS, immutable, et cetera, right? But DAX, mount root and auto mount aren't persisted. They're just, there's something that's... They have information from the VFS about the thing, the object you're looking at. Yes, they're manufactured from some other property. So, I agree, but anyway, it's an interesting thing. So, they're useful. In at least one case, so in the case of VFS, actually AFS and NFS and CFS, the auto mount may actually be persistent because it may be a property of the underlying file system. Oh yeah, so it's persisted, but not as the attribute flag, it's persisted as, so it checks some other thing and then sets this flag based on the other thing, right? It is effectively an attribute flag in AFS, for example, probably in NFS, I don't know, I don't remember. So, these are fine. It's a little bit confusing when you look at verity, like what does verity actually mean? It seems like it's a read-only file that has higher integrity. Yeah, I mean like verity means something very specific, right, it's the FS verity stuff. But FS-C64, F2FS, all support, it's like a, yeah, it's like a tree of everything. And it's a specific type of data integrity and it's a specific attribute that it's read-only. So it has these two properties. You could imagine, for argument's sake, taking an NFS file or an AFS file or an SMB file and marking it read-only and then setting some attribute that tells the server to do something strict. And that might actually have even more value because in the NFS or AFS or SMB world, you have more opportunities for corruption as you're copying it from here to the server room a thousand miles away. But right now, verity does seem very specific, very, very specific. Yeah, it's very specific. It's like on the order of eCrypt FS. So the Stadex flags that I would suggest, we consider, and obviously, we could look at other file systems to see, Beecher FS probably has some internal flags that could be added, but the four that just jump out at me that just seem like no-brainers are, this isn't an issue so much for me in SMB, but as I think about this, it's very common that people have local files that have underneath it, they're cached in Amazon Cloud or Azure Cloud or Google Cloud. And so accessing those files can be slow, but they're basically offline, hosted remote. That's a common scenario. And there's also the reverse where an operating system is told, do not take this file offline because I can't afford the latency to read it from Amazon Cloud or Azure Cloud or whatever. So please don't, now these are not really relevant as much for network file systems, we just report this. So for example, if you're mounting to a server here in this room and that server had files that were back to some cloud provider, that might be relevant, but it's not really something I have to deal with, I just report it. But there are lots of local file systems that have weird caching stuff that makes a hierarchical stuff off in the cloud or something. So offline and pinned are common ones. There are, like to take Windows example, they have additional flags that allow you to know more about what's offline, is it the data, is it the metadata? So there are other attribute flags in Windows and Macs and things, but the highest level ones they provide are offline and pinned. Those seem kind of like no-brainers. I can report these, NTFS can report these, and I'm sure there's others that can report it. Integrity and the opposite of integrity. This is some scratch file we don't care about, don't bother doing expensive checks, or please file system do the best you can for checks. These are kind of relatively easy to explain and the file system is allowed to do whatever it wants under that unlike Verity where you have a very specific merkle tree you have to use and a very specific algorithm. This is just a general thing. So in a Windows or a Mac, you'd write mouse button on a file and mark it. Just like the same you do for compressed or encrypted, you just click a checkbox that say, you know, so these are the four. I kind of question the underlying assumption that it's as simple as that. I mean, the integrity subsystem in Linux, for example, it's hugely configurable and there's policy and all kinds of stuff. And I'm not sure how a flag like this would be used. Yeah, so the way I would describe it is, so I'm not as familiar with how the backend of, let's say, Azure works or how Oracle Cloud or whatever works, but what I understand is that the take compressed or encrypted, when you select that flag, it's compressed or encrypted at rest. The algorithm used to compress it at rest when you select this flag is up to the file system to determine. So this is a hint. But why wouldn't someone always want integrity in their file storage? Well, the point is, this is not integrity. This is asking for the strongest integrity possible because this is the most important file in my system. These are set, you know, one out of a thousand files and it's asking for additional integrity checks on it. The opposite, like a temp file, would be no integrity checks. The 99.9% of the files or 99% of the files have whatever the file system is doing, which is usually pretty significant integrity checks. Sure, I just feel like there are so many shades of integrity that one could request that having a single bit is probably not gonna be useful. I mean, that's possibly true. I would like to see a lot more exposed about what is being requested and the applications I think need to have a little more control. Yeah, so the general thought I have on this is that we have hundreds of file systems available via Fuse, we have hundreds of file systems available via other mechanisms. I don't know all the integrity mechanisms they support, but we're asking the file system, do it's best effort and they can ignore it. So, in some way the main question is, what, which of these were the applications that you use? And so, would it be an application or would it be the admin? In other words, the admin provisions are a lot, you know, you start work at Red Hat and Red Hat provisions are a laptop. And on the laptop, they mark five or six key Red Hat security files with this tag when they provision the system. Maybe it's not an app. But putting this in statics is you're expecting applications to actually be using these regularly as opposed to putting it somewhere else where they would have to go and query to the site that's sort of like the Chatter, the Atter Flags in the X-T3. But offline is the main one I can see because that says don't touch this unless you need to and that's a really good hint for desktop environments. Don't go and reach the magic number on each of these because it's going to be slow. The rest I'm not so sure about. The file system might need to do about them because the file system might need to do something with them. And there may be requirements that we add some way for someone who actually wants to set these flags to actually read them and set them. But I think statics offline is possibly the only one I'd really put through statics because I said... Yeah, Steve, I think there may be an assumption that I'm hearing in what you're saying that I don't believe is true, which is statics is literally just for stat. There is no interface to set, say, statx atter pinned via a Linux system call today. This is only for statics is designed so applications can find out information about a file. There is no system administrator interface to magically set statx atter integrity. And so the question I have is what applications actually would need want to know would have an interest in understanding, say, that a file has the no scrub attribute set on it? I mean, that's a good question. So I hadn't thought about it that way. I thought that it was relatively easy to set it compressed and encrypted through tooling, but I haven't played with the tooling for setting compressed and encrypted. Yeah, it's not via statics. Right, so we should set some history here, right? Which is originally there was an IOC tool that was EXT2 specific that was used to set a maximum of 32 bits and also allowed you to read 32 bits of attribute flags and it was originally intended to be only for EXT2. At some point, other file systems said, hey, that looks like a really interesting interface. Let's hoist that up to the VFS and translate those 32 bit flags, some of which were read only, some of which were read right to the VFS. But the problem is that for EXT2 and EXT4, that 32 bit flag field is literally the on disk representation of those attribute flags. And as EXT4 started adding more attribute flags that had absolutely no relevance to other file systems, people started saying, wait a moment, why are we reading these attribute flags for which we only had 32, some of which were EXT4 specific, some of which were read right, some of which were read only. And so statics got created as a way for applications to read attributes in a namespace that was separate from the chatter LS adder I octals, right? But at the moment, there only exists an I octal to or system call statics to read the attributes. There is no system call that I know of that allows you to set file system independent statics flags in such a way that they can be plumbed through so that a system administrator could ask a file system, please set statics adder integrity, right? And this is where Chuck's concern comes into mind, which is that from a system administrator's perspective, you may wanna be, it's very, very file system dependent, what sort of integrity services might be requested. And it might be a hell of a lot more than just a single bit, right? It might be use this quantum proof crypto checksum, it might be use this all the thing, right? So statics is only for reading. But I mean, to be fair, I mean, that's a very good point by the way, but to be fair, we have a history in lots of operating systems, including Linux, of you provide an I octal or FSTL, whatever you provide an I octal, and that I octal has all kinds of details about what exactly the integrity, whatever, but statics is just cheaply reporting that this file is a... But that's the main point of statics. It has to be cheap, because we call it a lot. Because the other three offline are putting in statics. The other three things, this is what I was thinking of FS info for, is a separate thing where you can say, tell me more about, give me these other flags that aren't so commonly used, and also have another thing tells me, what does the file system actually support? Because it's really, all these flags are three states. There's, it's on, it's off, it's not supported. Yeah. Like, PIN is a good example. At least three states. I think most servers, most operating systems until the last four or five years, haven't supported the concept of refusing, you know, you can set an I octal or whatever, that says, do not take that file offline. And then there'd be an extension to FS info, which is the set info where you say, this FS info attribute, change it. Which is where, well, James is in the other room, I think, but I think we have something about this tomorrow. Where he wants to, a more comprehensive interface for setting and changing these things. Also, you mentioned encrypt as a thing. We've already got a good interface for setting and changing attributes. And it's called the extended attribute interface. Maybe we could define extended attributes for all of these and then statics could be the fast path for reading those extended attributes. And then it'd be set by the extended attribute interface. I mean, to me, that's fine. And that actually maps with some of what the max do, right? The max use extended attributes for a lot of these kinds of things, the Mac OS. But, you know, I'm fine. And if there's no way to set it in the short term, I'm fine with that. But what I'm worried about though, is that the first one I have so much like PTSD from years ago, thinking about, you know, if you don't report this, the nightmare you have is you have a GUI, some customer writes a GUI. And this actually happened in real life, right? So they write a GUI. And some of the files are backed up into the Amazon cloud and they open a window and suddenly everything stops on the system for a long, long time because the GUI wants to read the first 500 bytes of the file. And so, you know, that's something I personally remember having to deal with. So being able to advertise cheaply that it's offline, not the metadata, but the data. Yeah, so I think we've wandered off into the weeds here and it's lunchtime. So let's call this one table, move it to the hallway track. People on the phone, we will be back in an hour in the same room. Is this still on? Yeah, just want to say, Kent, Kent, come on.