 Hello everyone. I'm Paul Phillips. It's nice to be back since I was here giving this same well No, not the same talk, but the as the same label last year and If you are not tired of clapping for people yet, I have a rare opportunity to recognize someone John said that he hasn't seen his wife and children in six weeks. I haven't seen mine in six years, but they're here I'm meeting my younger daughter for the first time But it's true like a great deal of the work I've done on Scala and on this and on everything is only possible because of the Amazing patients of my wife and children who are sitting back there. So it would be wonderful if you would clap for them Thanks, that's great. Okay. All right six more years in the bag. All right So the first person I mentioned this title to immediately Intuited what the gist of it was but I understand if nobody else does so One of my favorite quotes has appeared any number of times compatibility is deliberately repeating mistakes The snake that's eating its own tail. That's us as we attempt to move the ball forward and always end up Compromising our new attempt with our determination to work with our old stuff and so if you have a snake that's eating its own tail and you find that to be an undesirable situation then you need the snake to spit the tail out and Most of you are younger than me at this point, which is kind of a sad observation for me to make Maybe you're so young that syrup of epicac is not something you've ever heard of it was already on its way out in my time But anyway, it was a way that you've made people throw up That's its purpose. I guess it's not used for that anymore. So this is very this I may as well be referring to like lot in them at this point, but Yeah, I didn't have a better idea for what it is that's like has a direct association with Vomit so because that's what we need we need to like really really break that connection and yet we're still completely dependent on Working with everything that exists in fact everything that fails fails because it asks too much of people Something's got to change in some way. That's too annoying and it doesn't take much to cross that threshold So the the gist of this my new idea the whole thing that I'm throwing myself behind is a mechanism for Making things better without losing the things that were worse and as functional programmers You will especially appreciate it because the ambition here is for our lives to be functional programming so rather than having these like ungodly parades of non-determinism and Mutability around us in which we try to carve out this little section and write functional code and be happy with that I'd like it all to be the kind of the have the good functional qualities that we seek and I thought of a way that I think Makes that possible as long as I hand wave a lot about what exactly it means Since a some percentage of you have no idea who I am. I feel I should always include this credibility slide Now importantly that should not actually give me any credibility if anything it might work the other way But if nothing else it means they didn't drag me in off the street to give this So that as you can see like the size of the bonfire the Scala bonfire over time During the period in which I was working on it is very similar to the size However, you also notice that the bonfire goes out in 2014 and so for essentially two years Somewhat less but not a lot less. I've been looking for like what else I can do what to do What do I throw my like what can I be enthusiastic about and this is it? I found out of the like essentially at the last minute that I had 30 minutes now like 15 I guess since We started a little bit late. I'm assuming I'm not going to get a hook at nine on the nose But I do have an enormous amount of material and not like padding material But like legitimately, how can I possibly leave this out? I because I cannot communicate like sort of the scope and the breadth of Possibility here without mentioning all these things. So I got a talk. I'm in the miscellaneous track but That's okay because I didn't want to be in the advanced track because it isn't advanced per se on the other hand I don't want to be in the like, you know, I I just woke up track because then the advanced people are like Oh, that's too simple. It's the very cross-cutting in its potential audience. Everybody should come to that all the other rooms should be empty because That's just my enthusiasm for my subject so and hopefully in 15 minutes from now you will all agree and Although epic act for the Arboros is the best title. I will ever have for a talk the virtual file system This changes everything is actually probably second place Although the why it's funny might not yet be apparent. So I actually will have to come back to that all right, so what this is suffuse is stripped down to the Trivializing core is just virtual files It's more about the consequences of having virtual files available in a pervasive way and then What we could build that exploited that To improve the rest of our lives so my not so grandiose claim here is Transformed software development practice And so but if anybody who's actually observed me in my time in Scotland knows that I'm actually not really keen on making grandiose claims It's very Antithetical to my nature. I prefer to undersell than to oversell so I'm mentioning that to point out that like the fact that I would Say something like that means either I'm delusional Which is always should be kept open as a possibility or or I am genuinely on to something here in which case It would be great if you would stick around and hear enough to be convinced or to commit me maybe if delusional wins and Then this is an acknowledgement. That's actually it's left I would have made it smaller, but then you couldn't see any green the pixel Opportunity the last slide but essentially the minimum But the point here is is like I can just scratch the snout of the crocodile in terms of what's possible to do with these powers and so Because of whatever example I pick you'd be like oh well That's not something I need or like that was trivially done 20 years ago Whatever you know you have some mechanism for dismissing it if you're so inclined just remember I could tailor an example to you that would excite you just come tell me what you need and I'll tailor it so We need some definitions. I won't unload websters on you, but just so we're talking about the same things You know what a path is or at least we have some intuitive notion of what a path is Something that we can sort of build up incrementally or at least prefix match against that has a Restricted form perhaps like slashes are not allowed in simple names, and it's assembled by joining slash whatever, but it doesn't matter Because this is the level at which we're going to define it a file has the very specific definition that it's whatever you find in a file system That the reason it's defined that way is so that we understand when I say file I mean anything you find in a file system which certainly includes directories symbolic links character devices, Pifos sockets Whatever The point is that there's a thing out there. So sometimes when I say file. I mean like sort of the classic pile of bytes It should be obvious from context if I mean that specifically or this it's hard to avoid this ambiguity One tries to pick like one or the other and then finds that you're just there's a blizzard of different ways of using it You just got a deal a little bit. So generally though file is generic Now now that we've defined files so specifically that you that it's clearly defined a file system is something that maps paths the files This is not quite a circular definition. We do have one little anchor here path leads us back to string, which is a real thing Metadata also super vague, but it's what a file system knows about its files The reason metadata is important is that we need a two level sort of evaluation strategies such that There is different laziness in metadata and data. This is absolutely critical to Exploiting all the wonderful myths of virtual files because it means we can have metadata without yet having data and Thus we can construct Imaginary file systems around the metadata that do not need to actually be delivered until the time comes Which is like lazy evaluation. There's lots. There's a whole bunch of nice parallels between The programming language world where we have these things and the file system world where we could have these things if we have this so I've been vaguely disappointed at my attempts with individuals to communicate my enthusiasm for this idea as they heard my words and said, yeah, all right and Then you know, we're ready for the next idea or whatever. They weren't like yeah, man. That's the thing so either that's like my inadequacy at communicating always a possibility or That it's harder than I think and this is also quite likely to See what I'm talking about because the assumptions are so embedded that even if you think you're discarding them, you're not This is you know quite reasonable We're really used to thinking of a file as a thing that's already there that you can go look at not as a program that is run When certain an open or read any sort of file system oriented call takes place so you must work really hard to appreciate the goodness of this by forgetting everything you know about files and thinking of every single thing that happens in the file system as part of a big running program that is your life on the computer and Suddenly you have like aspect oriented programming You have all these places where you can inject code and get in between point a and point b and make something happen or just change the behavior of things in a completely transparent way or have Divergent behaviors for the same thing depending on what is coming to look at it You can give it exactly what it needs But only have one source and this kind of if you the amount of duplication that exists in a typical file system Is off the charts, but it's really unnecessary right like if you put your whole file system in a database Database to be like yeah, you got five copies of this Ten gigabyte file. Let's just keep one in the database and give it to you in five different spots We don't need five copies file system can't do that. It's just like all right, man You want to burn your disk that way. That's your problem So once you have all these new hooks into your life You have suddenly turned every program into a program that you can Radically change the behavior of if necessary that you can feed exactly what it wants and that you can Reprocess exactly what it gives you into exactly what you want and all of this without any change to anything And so this is kind of amazing like take something like I don't know Like Picasso so Picasso does all this stuff and it reads files and it writes files And it does it all in a way that's super annoying to me Like I want 95% of what it does, but I wanted to do some other stuff completely differently, and it's really hard there's no mechanism in Existence that allows me to just say like I want the file system to seem different Both in the way that that program reads it and then I want it to expose its idea of things as a file system That's completely consistent that allows me to see exactly how it is that it understands things For instance when you do a bunch of facial recognition in Picasso It builds up this whole database of faces and whatnot Well, I'd like a file system that just had the names of everybody I've identified And that's a directory and in that directory are all the files the pictures with them in it Zeroed in on their face or whatever this would be extremely convenient for me all the data is in place for it Yet for me to do that right now without something like this I have first I have a big old like one-off job to convert it and then I converted it and it doesn't stay up to date It's just a snapshot of life, but I want it to be always there I want it to be implied and this is possible now So we have this notion of data flow programming The spreadsheet is the usual example You've got a thing and you've got cells and they depend on each other in whatever way you change one thing You don't have to go and say like okay now update The cells that depend on this it just does it that's how it works It knows I depend on these other things they changed I change Well, why do we settle for only having this in this tiny narrow domain? This is obviously better right if you have a file and they need to be related in some particular way Why on earth would you want to be tasked with updating them together? Right, this is nuts. One of them is strictly derivative of the other But in order to live like that we need a mechanism for the second one remaining a reflection of the first one this is that mechanism and So I call them reactive files because reactive come on I mean But it really is like the best term I can think of to make the point right so there there It's a reactive file and it just like it just reacts to the changes elsewhere. It has no independent existence So here's another little buzz phrase you can listen for in the years to come as this becomes like a thing This will be a whole like oriented programming Continuous consistency This is like the way that continuous delivery became a thing continuous integration continuous consistency Well, let's not allow a window of time between when a change happens in one place And when all the other things that depend on it change or not worse yet Not just a window of time or possibly it's lost and the update doesn't happen or one way or another they get out of sync It's all nuts. We can have continuous consistency such that file B Which depends on file a is not something we need to go and update after we update a B is a function of a It's defined that way in the slide if you're not familiar with the notation Colon equals is supposed to suggest assignment But just equals is supposed to suggest Mathematical equals as in the left side on the right side are the same thing So this is a giant Difference in more ways than can possibly be enumerated for the implications for reasoning about things for what you can expect to be true For just like what's the right way to do it and the right way to do it is the second way It's only from the inability to do it that way that we don't but we can have the ability. It is available to us in short order and of course Naturally B is cached and smart and so only computed if necessary and This can add up to a lot of saved effort if we're perpetually needing to update B Just in case something comes along to read it But why we can wait until something comes along to read it and then update it This may save us orders of magnitude of effort if it's something that's infrequently read most things will fall into that category All right, so you've had your hints now. I bet you can reason this out What am I talking about? It's immutable statically typed has type inference lazy evaluation unbounded in size hand wavy a little bit Concurrency safe also hand wavy a little bit, but not really like you know to the extent that that could mean something in such a short phrase It is What is being described So yes, that's very spooky notion All right So go back to the slide in your head where I said like you got to on unmake all these assumptions that you have like Foremost among them is like this is nuts, right? That's not how files are you think this isn't how it works That's not what I learned. Well, I think this is how files could be and should be sources of inspiration so as we all know There's all kinds of good papers all the time that we're all ignoring and any time you write something You'll go eventually you'll find a paper written 30 years ago that outlines exactly how you should have done it and There's it's too late now because millions of people are using it and it's terrible We've all played this out a billion times. We're certainly not done But even you know, even though there's all this great neglected work in computer science stuff There's even more in children's books to my Experience thanks to my children. I know about it in this particular case There is a book called a house is a house for me And I have a few excerpts from it, which I think indicate that Marianne Hoverman was way ahead of the curve on this matter So I'm not going to try to sing this for one thing It's not a song but it feels like a song but I'm not going to sing So instead I'll just say it in a in sort of a vague monotone Cartons are houses for crackers Castles are houses for Kings the more that I think about houses the more things are houses for things. I Can testify this is definitely true as soon as I started viewing the world as like file systems Everything was either a directory or it was something that I would put into a directory And I couldn't see the all it's everywhere. I look houses is for things By the way far fetching is being used as a verb here, which is fantastic Perhaps I have started far fetching perhaps I am stretching things some a mirror is a house for reflections a Throat is a house for a hum. But once you get started in thinking you think can you think can you think? And If you get started in thinking that it's actually a better transition in the real thing, but I skipped those some pages I think you will find it is true that the more that you think about houses for things the more things are houses to you See because you already know that I think everything is a house And once you get started in thinking this way it seems that whatever you see is either a house or it lives in a house and a house is a house for me and And then there were these lost lost verses that do not appear in any published copy But I believe are that my personal property that I'm revealing here for the first time a Jason's a house for a map of maps a map is a house for its keys Each key value pair is a newly built layer. This continues as long as you please That's an infinite potentially infinite file system right represented as Jason of ours a house for a bug a bug a Thread is a house for deadlock, and it's a safe bet that your technical debt has the nicest house on the whole block I actually had like six verses like this and I kept it to two so you're you're welcome and If you're inspired by Mary Ann Oberman, she has many other books that I think are applicable. How do I go? Bugs now the first two have very obvious subject matter. The third one is compiler architecture. Mr. and Mrs. Muddle The fourth one is about dry One of each and the fifth one is supposed to be called all kinds of type families But it's assumed that you understand that that's what that's about All right, and since I had to just ruthlessly cut every like concrete example out of this You're just gonna have to take my word for it that it's actually slightly less hand wavy I do have tremendous amount of material and now is going to be your chance to you know leave all those other rooms vacant I'm in 206 Coming up next container transparency some exciting terms typed files immutable files Meta files lazy files implicit files and reactive files a subset of which we have defined a subset of which you can Intuit a subset of which I just made up Virtual paths which is to say where we're gonna reverse the notion of what dictates what and have the path be a consequence of the metadata for instance Your camera gives you all these things called image underscore one two three four five I don't want to have it by my job to run around renaming this stuff How about you just take whatever's there and give me a file system that's beautifully organized by exactly the time format I want with the images sitting there with whatever time-oriented name or their faces right whatever just let the Metadata dictate the name of the thing and let me don't make me do your job computer if it's called image underscore something That's what it's called. It's your problem to give it to me in a nice way There's a whole universe of cool stuff involving well if a process can expose itself in a file system on disk Then you can that's this is a serialization mechanism So you expose what's interesting there and now our sync allows you to move a process to another machine and resume it over there Now this has a whole bunch of hand waving associated with it as well. Yes, but it's completely legit You could actually do this so and on and on and on and I think since I sensibly anticipated that 30 minutes wasn't very much This is the end of my slides for this, but again, I'll just emphasize one more time There's lots more coming up in just a moment, but that's all I have for the main audience