 My name is Seth Jennings, I'm a senior software engineer with Red Hat, I work on OpenShift by day and this has nothing to do with my day job, this was just something I did for fun. You may have seen my video online way back in the day when Bash on Windows aka Windows subsystem for Linux first came out and Ubuntu was the hard-coded option for that. I used Ubuntu to bootstrap Fedora into that environment. It's gotten a lot easier these days and now you can deploy them through apps in the store. So this is just going to be a presentation on what the Windows subsystem for Linux is and why it's cool and why it allows us to reach a lot more developers with Fedora which is kind of the overarching thing here. So this is the rough agenda. So NT was originally designed to be able to run POSIX programs and actually it was designed with kind of a core kernel with subsystems that would handle different user space APIs and POSIX was one of them. But they were all user mode implementations and required you to recompile your binaries. They didn't run ELF stuff. And so the original one was the POSIX subsystem. The one that I recall using is the Windows services for Unix that you'd go into your Windows software center thing and check the box and then you could run a limited number of Unix services. And then SyncWin is also a popular one but it's another user mode emulation thing and you have to recompile the binaries if it has its own package manager and things like that. So there's a lot of work involved there and the package selection is always going to be limited because it's limited because people have to port any application that runs in those environments. So yeah, they didn't run native ELF binaries right and so there was always porting work involved. There was always recompilation required. So the Windows subsystem for Linux is different in that it is implemented primarily in kernel mode and really what it does is it emulates the system call table of the Linux kernel. So the API that it is presenting to Linux processes is a kernel one, not a user one. And so that has some interesting properties. So I'll just read off the place here. There's Peco processes which in Windows Win32 is the primary user space format for programs and it's amazing that going to doing our research for this presentation the NT kernel does tons of reaching up into user space and assuming things about the structure of our programs and user space which Linux really does not do. Everything is calling down from user space. Really there are limited mechanisms for the kernel to reach up into user space like signals and things like that. But there's lots of registering handlers in Windows. So it's got the system call emulation layer but there's very rarely a one-to-one mapping between these calls in Linux and the calls in the NT kernel. It's typically some combination of calls in the NT kernel and in a lot of cases they have to implement features that are in the Linux kernel in the kernel mode driver. Things like the virtual file system and Berkeley sockets, things like that that the NT kernel just cannot handle natively. They've actually written quite a bit of code to emulate those in kernel mode. So back to the why though. Primarily for the advantage for Fedora is Mindshare that people running Fedora on Windows are not going to be using this for production workloads and you'll see why in the demo here in a minute. But basically these environments are pretty transient and you have to have the terminal window open. If you close that terminal window all processes inside of it it's essentially a shutdown of the LX instance, the container environment in Windows that is running the Linux processes. And the Cisco emulation table is not complete although it is shockingly functional, almost any program that you could run will run okay in that environment. But there's developers that have Windows on their laptops and those laptops are issued to them by a company. They can't dual boot then they can't reformat them with Fedora. And so this puts Fedora within their reach for development which is really good for us. So we're not really at risk of losing any Fedora users to Windows but we can gain Fedora Mindshare among the devs that are voluntarily or not stuck on Windows. So to talk about how, why people looking to run Linux binaries on their OS target the system call interface. So this is the nursery rhyme that we all learned as kids, right? It is the system 5x8664 ABI calling convention. And you know it's first you start your arguments in the non volatile registers. But this is basically how this works and the calling convention is, some of this is specified by x86, obviously the syscall instruction, the sysreturn instruction. And putting the system call number in, actually that's part of the calling convention. But this is a very well defined thing that can't be changed and that's why people like to target it because they can depend on it to always be the same. And that whole calling convention is part of the Linux kernel ABI. And that is the thing that if you break, Linux will curse you in public, right? And so the system call arguments and numbers, they're all compiled into the L binaries. That's why you can't change these things because if you change them, then binaries that you compile with the old headers won't function anymore. And so that is why the keeping a consistent ABI is so important. And so people looking to emulate the Linux kernel and usually target this layer. Microsoft's not the first person in the first company to do this. Alumos did the same thing with LX branded zones, which are kind of like containers. But they are containers with a little bit of metadata saying that the processes running in here are Linux containers and you need to emulate their syscalls. So there are lots of system calls in the Linux syscall table. There's over 300. And that is really not a really good number because there are tons of combinations. If you've ever seen like set sock op, there are tons of socket options that you can set. And same thing, a number of system calls have a flag that you can pass in that basically creates the function operates very differently depending on the flags that you pass in. So there's lots of cases to cover. And Windows subsystem for Linux has prioritized these based on profiles that they've run of commonly used Linux applications. And so they're adding support for more as time goes along. And there's people filing bugs and things like that. So yeah, so I'm trying to go through this technical stuff because not everybody is interested. And I wanted to do a demo because that's pretty cool. The core component is this lxcore.sys, which is a kernel mode driver that Windows loads. And basically it registers with the NT kernel and says, if anything starts these Pico processes, which are these special class kind of black box processes that the NT kernel doesn't know anything about the structure amount. It says, if you get something that comes across the syscall boundary and it's coming from one of these Pico processes, pass it to the lxcore sys driver because it knows how to handle it. And it also implements something similar to the Linux VFS, the virtual file system in the kernel, to provide a translation layer between NTFS and a POSIX compliant file system, which in this case is called VALFS. And basically what it does is it stores all of the POSIX attributes and extended attributes in the NTFS, in the underlying NTFS file system, and kind of projects that file system into the Windows subsystem for Linux from the instance. Now that translation is not portable and so Windows can't really do anything with those files. If you were to go into the directory that holds these files in Windows and add one, that file would not appear in Windows subsystem for Linux because it doesn't have the POSIX extended attributes and the file system, VALFS doesn't know how to project that file into Windows subsystem for Linux. So that's where the second file system comes in, it's called DriveFS. And basically mounts the drive letters in Windows, so like your C drive, D drive. It puts those in slash mount slash drive letter by default. And that allows interoperability with Windows. And so any files you create in that mount point in the Windows subsystem for Linux appear in that location in Windows and if you're to create files in Windows there, they appear in your Windows subsystem for Linux instance. Does VALFS require like a dedicated file system for it? No, it's overlaid on top of NTFS. So if you create a file under VALFS, will it show up on Windows? Yes, it will. Just not vice versa? Right, not vice versa. Because when you create it, when it writes through down to the NTFS it puts those positives properties in the extended attributes on NTFS. But if you create it in NTFS it doesn't have those extended attributes in the, and VALFS doesn't know how to project it into Linux, right? Because it's like, I don't know what the permissions are on this. So how do you bootstrap that so like you can write your libraries out in such a way that you're executable remotely? I didn't understand the question. So it's like, how do you, you have to bootstrap your VALFS so that you have enough of a file system to like run something, right? Right, so that is where the installer works. So it actually creates a small bootstrapping instance when you install the Fedora application. So I'll show that here in a second, but when you start the Fedora installation, there's a basically a RuneFS tar ball in there. It starts a bootstrapping instance and then expands that it basically untars it such that the positives attributes get written. So that's how you bootstrap it. Yes, they do. They kind of do some hackery around networking stuff where they've got a user mode session manager that will figure out like the network interfaces and the DNS server and stuff like that. And they actually inject etsyresolve.conf into the environment themselves, using an init that they also inject into the environment. So this environment does not, I haven't found a way to run system D on it yet. It basically injects the init and then calls a entry point that you define when you create the application, which is kind of the entry point. So that's actually what this slide is describing right now. So it has init, part of init is talking to the session manager to inject the DNS servers and then the networking adapter stuff is actually, when you do IPA, it's actually making SysCalls to figure out the network interface information and so those are emulated by the kernel. And the IP address that you will see in IPA and Windows Assistant for Linux is the IP address configured on the Windows box. And they actually share like port space. If you bind to port 80 in Windows Assistant for Linux, a process in Windows cannot bind to port 80. So yes, so the lifetime of the LX instance is bound to the lifetime of any processes running in the initial shell. So if you exit bash, the whole thing gets torn down in any programs that you were running in it in the background, Fortner on whatever, go away. At the same time, if you open up, I'll do this in a minute, but if you open up a Windows for Subsystem window, if I type Fedora on the CMD line, and it drops me into Windows Assistant for Linux, I can start another window, type in Fedora, and it puts me into the same Alex instance. So if I do a PS, I can see the other bash instance running in the other window. So this is a graphic kind of displaying what that whole thing looks like. So this is kind of an older diagram, so it says bash.exe. That's when Ubuntu was hard coded to be the thing. But now we have Fedora.exe. And what that's gonna do is make a comm call, which is like debuts in Windows to the Alex assistant manager, which will make an iOctl call to the kernel to set up this Linux instance environment. And it's very similar to a pid name space in Linux where processes in this are isolated and they're kind of flagged as these Pico processes. So that the nd kernel knows, I can't handle syscalls from these processes directly, I need to pass them off to this Alex core driver. And so the kernel will call init and then go to bash. And then bash will actually make straight system calls to the kernel. And there's no user mode emulation there. So demo time, I don't have time, I might be burning through this. 14, that's good, that's good. All right, so you'll see here on my desktop that I've got this Fedora AppX and then for that to open that, it's already installed. And so this is kind of what it looks like. And it's just like a Windows Store application for Fedora, right? This is, there's so much wrong with this. Did Microsoft create this? No, I didn't. There is actually installs Fedora on Windows and when you reboot. That's right, there's no GUI, it's just a command prompt. They've actually, like I said, Microsoft really has done well here in that they made a distribution, a generic distribution launcher, AppX package, that basically any distro can create a tar ball, icon and assets, change some packaging metadata and then type build and it just, it's just magic. So that's what I've done in this situation. And the tar ball that I use, I just did DNF install root and installed a little bit more than the Docker image but less than the cloud or server image because you don't need the kernel in there. But I wanted more than Docker had to provide a better out of the box experience for developers because that's going to be who's targeted at. So like SSH tools and Git and things like that. The main pages. Yes, and the main pages. There's lots of stuff to strip out of Docker that makes it not so great. But then all the other composes have the kernel in it. So it's kind of between composes and that might be work to do to create a custom compose for this. Once we work out the other kinks, the non technical ones. Which is why this isn't in the store yet, I can't say anything about it anyway. So if we hit launch here, it's going to bring up that and I'm in. Actually, I'm not sure if I can. So then terminal emulation here, right? I'm not sure if that's going to scale the whole point. Can you right click on the doing? Yeah, that's what I just tried that. And they're going after the next generation. Yeah, that is as big as I can make it. I think I don't want people to take to resize it. A virtual, virtual terminal. But I guess one of the things that I was wanting to show is you can go in here and you see C is mounted under slash mount. And if I go in there, that is, yeah, I don't have permissions on those. So that's another thing to point out is that you can be root inside this environment, but you are not administrator in Windows. You have the same privileges that you have as your Windows user. And so if your Windows user doesn't have privilege to read a file in C colon, then you won't be able to, even as root, you won't be able to. I can do sudo ls, actually, it did stat it on that one. But I probably can't, I can't go into slash windows and mess stuff up. And if I go into user, yeah, look at that, there at the desktop. If I just, you know, echo interop into, I just created this file on my desktop, right? And I can open that up and it's got the contents in it. And to demonstrate the shared port space, excuse me, I work remotely and I've said more words to people in the last 24 hours than I usually say in a month. So let's see if I can remember how to do this. That's skstart, is that how you do this? Yeah, sure. Yep, all right, so we'll do that. Yeah, I did this before. So these are the kind of problems that you'll see, right? Is that you'll try to open, set some esoterics socket option or something like that. This didn't work for them. Although I didn't try this, execute it in that way, maybe I just do it like this. No, that didn't work either. This part, I'm gonna link. Could always do a, do you have Python on there? Yeah, Python-m, yep, Python-m-http.server. Yep, you knew what I meant. The vapagee is trying to be smarter. What is the month? HTTP.server, no, that's right. Well, I need to interrupt like text again. All right, just HTTP.server, 8000, all right, so let's see if we can get this done. So interrupt.text, I'll try to hit that file from over here. Edge, it feels bad, doesn't even focus on the Andos bar, all right, 1000. You can't just sit in Canvas on 4th solar in 1028, there, you can't just sit in Canvas on 4th solar in 1024. I didn't do pseudo, let's say that makes a difference. Yeah, something was in the process of actually getting that done. Oh, was it? It may be that I'm running this on DriveFS, let's see, as long as we're gonna use hdbd, it's something native. Wait, it didn't eventually pull it down. It's not basically enough. Edge, I don't need to have to open this text file. It's just there. Sure. There's no interoffice I can't get into the one you made here. Oh, that's right, that's true. There, okay. So it's in the same board space, right? And I guess one more thing, if you, so if I do Fedora again, and this is interesting, if I do a PS over here, then you can see it's all in the same process space, right? I'm basically open to another terminal on this Alex instance. Yes, I haven't installed anything yet. No, you have a T in there. Oh, oh. I'm pretty sure that they do support, you know, ACLs on this. Oh, no, no, no, there's, I'm not sure what it would show on ballFS. It's almost trying to figure out how he can hack this. But it's, you know, similar to, I guess, Samba or something like that, where a lot of the POSIX fidelity isn't there, and you kind of have to. Yes. So how, I have two questions. First of all, how, so how tightly is, like, LX Core tied into, you mentioned Batch, Colleen Batch a lot. Is that an artifact of the Microsoft repackage set of packaging? Like, if I wanted to, say, run, like, FISH as my, you know, as the main application and not have Batch in there at all, because I don't know. You'd have to repackage it. Okay. Because that entry point is defined in the, in one of the packaging files. Okay, so it's not, okay, it's not somehow. You can't pass it in as, like, a flag to Fedor. Yeah. And the other question, so is, how does, how is the terminal emulation here interacting with, like, is it, do they just implement a terminal emulator here and it's interacting normally via TVYs, or is there? They, I've heard, and I should have installed this. I've heard that the terminal emulation is good enough for T-Mux to work. Okay. And it's actually worse. Yes, yes, they are improving it, and T-Mux didn't always work, but the rumor is that it is now, and I haven't confirmed that personally. I mean, you know, clear words, so it is aware of the dimensions and stuff. Yes. Yes. Okay. Yes, I was waiting for that question. Yes, so back to the X Windows thing. They can't, the way I've seen people do it is they'll run SIG Win and the X server inside the SIG Win and then run Windows subsystem for Linux and then use X11 forwarding and bounce it off the SIG Win. There are actually native Windows and servers, too. Yeah, there's a little BX server I've done custom build to base. Yeah, that's how you did it with SLA. Okay. There definitely are ways to do it. Microsoft definitely doesn't support those. Yes. And of course, using TCP software for a second question. Oh, yes. And we will be available. That is a good question. That is still being worked out. I'm, yes, we're toying with the idea of putting out the Apex and just the certificate that it signed with but people that don't care about the steeper curve can at least play with it. But yeah, there are non-technical issues that we're having to overcome. Yeah. Yes. So what are they targeting this for? Is it Docker or is... So you can't run Docker in here because all the clone flags don't exist and C groups don't exist. So there's lots of stuff. I mean, they'd have to implement a lot of internal functionality in order for namespacing and C groups to work in Windows like a prohibitive amount. So, you know, there's not really a threat to run Docker in this environment. I'm not sure that that's true. Oh, well. Yeah, but at some level, you're going to need namespacing, right? You're going to need Windows. You would install Fedora subsystem for Linux or sorry, Fedora and then install Docker from Fedora and expect it to run with Windows version of RunSeq, right? Yes. I mean, this could be used in theory to do... So right now they don't support multiple LX instances running at once. But if they did, then you could, you know, a Docker process, Docker.exe and Windows could create an LX instance and run a Linux container, right? You could run multiple of them and you could do that. But they're not that point yet. Well, I would just like to point out that Microsoft has a very strong incentive to make Docker work on Windows. Yes. I would agree. Well, it already does. They demoed a Docker combination. But I was just going to point out where... Yeah, fine. Okay, yes. There you go then. I can use your question on point. Sure. I'll do it. Thank you.