 So the first presentation is by Aaron Levin and he's going to talk about dual booting Yeah, okay So I'm gonna make this very fast because I have to go look at the couch But then I'll be back to join you all for drinks and dinner later, but so my name is Aaron I have you can read this but I Living here from Canada now What I want to talk about was the state events of dual booting a brand new MacBook Pro with Nix OS So right now this MacBook you don't have to believe me because all you see is a browser, but it's running Nix OS I guess I can do like you name Dash a so it has a Nix on there. So Because you couldn't I don't have to believe me for that either But yeah, so I want to talk about that and what some of the issues are and there is one major bug that may or may not be a kernel bug and I Will show it to you and then also I have a promise that if any of you wants to try to fix it I will donate 50 euro to the Nix OS foundation and I'm like a very Cash-conscious person. So that's like in my mind like thousands of dollars. Okay, but because I'm Consider my oh my god. It's not what I want at all. Ah, okay, so But I also have a meta presentation, which is a new presentation technique that I invented called tab driven presentations so It's really good. I'm doing it right now. I made my presentation two seconds ago. So That's why I'm a thought leader Okay, so the first thing I want to talk about is the level of support in the Nix community This is a issue. I had when I was installing Mac OS, I don't know what I'm doing in the world of computers And so I had this big issue where I just cut and paste it as much as I could and then these really kind people just Wouldn't leave me alone and they kept helping me and then I was like up like so late and You know, let's see what happens. There's no luck. It was like four in the morning I wish there was some kind of weight like it just goes on. I sorry Just really goes on and on and on that they were helping me with this issue and The good thing is is it hasn't see there's a green light here. So it had a good ending So I just want to commend the Nix OS community for being so supportive. Thank you Yeah, but so what is the the the current State of the Union if you want to dual boot a Mac OS with Nix you can do it it will work And it's a fine system and you get the good hardware and you can look really cool in the cafes but There is one really big problem and that's there's this Nasty bug here that says unreliable suspend and wait on the MacBook Pro This is 12 one, but if you scroll and scroll and scroll almost as much scrolling as my Nix OS support You will find many people are having this problem. And so I promise I would demo the problem This is a different screen, but also on the same computer and if I close this it probably actually might not break because of this business and So then you go into your work meeting and your boss is showing me the new deployment system on Nix OS and you open it and Nothing will show up And so you will panic because your boss is now thinking twice about all the new deployments that up you have and Then it'll start getting really hot. So now you're like sweating in the room with your boss because the computer is overheating and eventually it'll shut down which is good And then you'll reboot and your system will back up again, but I Mean you don't want that. So that's the the current state And if anybody really wants to like dig very deep into this bug I will lend this computer and we can hack away at it and try to figure it out and and I will also commit to sharing the learnings of how we debugged it online as well but otherwise Yeah, you can install it. It will work the suspend resume bug remains and Thank for the thank Nix OS community for the great support and that is the end of my talk. I Won't be offended if you don't ask questions. So don't feel that you need to but I will like leave this room and feel sad I'm only kidding help So this bug the screen does turn off but the system works Well, it's different. It's so I tried to like cat Two files and mash buttons and do as much as I could to figure it out. It's if you look at the logs System D registers asleep, but it doesn't register awake But when I open the the screen Like something's running, but like a few things. I noticed is like I have one of these Keys that the security people at SoundCloud made me use Ubex or something. I'm not a security person But the light won't like the USB is no longer gets detected anymore So like it'll be it'll detect it when it's shut or something, but I open it and then it shuts off So it's much beyond my understanding of systems. So likely something else But sometimes I get similar. I don't even make book But it I don't get the screen is black and if I switch to virtual Terminals and back to ex then it it's gets back. Yeah, otherwise. I'm unable to work around it Yeah, Matt. Yeah, you it straight up is like a brick and Mac Mac books have this really nice feature that these mouse pads have haptic feedback and So you really know when your machine turns off because you can't press the buttons anymore Which is good for debugging a black screen. I Mean like it was like literally I would be like is this thing even on like I can't hear anything and then like people are parting out because I'm doing it at Burghine all the time so I can't hear anything and The haptic feedback trick was a very helpful for figuring out what's happening Okay, thank you Next up it's going to be Matthias Bayer who is going to talk about a Collection of tools he's built called Nick's scripts. Hi So when I started with Nick's I had this one problem. I I added my configuration that Nick's and Re-edited and rebuild the system and it worked and then I want to go on and edit it again and again and again and No, I don't have don't have a device Don't have things to show So and I wanted to have the possibility to go back in my configuration.nix to a point where my system builds, but I Wasn't able to find it in the git reverse in the git history because there were so many comments So I started to write a script which Generates a Tag in my repository every time my system rebuilds So and this was the first step and then I continued writing scripts and scripts and I put them together into a collection of scripts and there are tools available for updating package definitions in a next package repository by pulling the patch from the monitor.nix.s.org tool Applying the patch trying to build a package and if it works push it to github and I just have to push the pull request button and everything is fine and I have exactly one step in my command line to update a package. I edited other tools to update my channels in one step so I can update my channels and If I can rebuild my system with this channel, I can go into my configuration.nix repository And I see I build generation 42 from this commit and from this channel at that time So I can go into my configuration Set my channel generation to this specific point and rebuild the system exactly how it was with this one configuration.nix Yeah, and there are some other tools available and you can find all the scripts on github and Matthias Bayer slash nix as my dash scripts Yeah, and suggestions are welcome So that's all Thanks Can you say for one question or two? One reaction or two? Jan? Hey, I suggest you tweet the link because your name is difficult to spell for some people Okay Yeah, I can Yeah, I can put it on your mailing list or whatever Yeah Okay, very practical suggestion very good next next question I went to your scripts because I wanted to do a difference between my two generations and just remember What's the change between generation two and three for example? And I also made a pull request to your repo so you saw that but we are very limited in We cannot do difference between generations because most of the information is lost So you do not know what are the difference between the nix scripts that were used to generate the two generations You know what I mean? Not really no if you compare to generations You can you can see what are the different different package that have been installed and you can look at the different ash So there is a package with the same name, but the different ash you can assume that is There was a change somewhere, but it's very difficult to know the root of the change Okay, so I know maybe you should talk about that. Yeah, someone has an idea of how can we retrieve the difference Between the two nix expressions that were used to generate the two different systems. That would be interesting Okay, you really should talk Have you thought about contributing any of this back to nix itself I'm wondering if maybe there's like you could have a hooks system or something in nix that would maybe hook into your scripts Rather than having to call your scripts directly would be a nice idea. Yes I think the purpose of nix and and my scripts are not really the same nix is like For me as a end user package management and so on and my scripts are like the surrounding parts But yeah, would be a nice idea. Yes One more time for one more. I can show you GitHub Not really a question just wanted to report that I'm a very happy user and what to say. Thank you for writing it. Thank you Okay, that's it Okay, so while we are setting up the laptop I'll introduce the third lightning Which is going to be by Emory Hemingway Who will be talking about what happens when you point nix to? G-node So I don't know what G-node is so I'm really curious about what he's going to say Okay, hopefully it works. Yeah, so I've been spending the last few months porting Nix to G-node Which is a microcordel operating system? It this is all experimental and requires features that aren't in the master G-node branch, so it's not really reproducible right now The goals for the port were I had to do everything natively I didn't want to sort of Do some do you next emulation? I didn't want to like just sort of try and fix system calls. I Wanted to be pure at least as pure as the Unix implementation. It's actually more pure and I'll get to that I have to have a lockless store because our file system doesn't have locks No database either That's something that may not work long term, but And it has to be invisible to Applications and that does work and I want it to be recursive, but that's that's theoretical and I don't see why it shouldn't be but that's not known So just I got to talk about Unix first. It's really old. It's only made for development It's it has a file system as intrinsic feature. It's provided by the kernel the file systems everywhere It's for multiple users policy is administrative by the root And you have users in groups Geno, on the other hand is released outside of the University at Dresden in 2008 It's designed to be low complexity high security Which makes things kind of complicated even though it's this little complexity It's kernel agnostic. So It runs on micro kernels separation kernels and monolithic kernels and I'm running a l4 family hypervisor right now and If it's going to count for anything we have to have legacy software support. So we have a libc We have SDL Qt and we have virtual box running so What we don't support natively you can virtualize we have distributed policy. This is we don't have a root user. So You can you can subdivide the system and then make meaningful policy decisions at different points and Then we had different services and all the services are addressed by capabilities and we have you know We can share memory using capabilities and threads and stuff like this But the file system is optional and that's where it gets tricky So everything in G note is done with the parent-child relationship Like and then you have services routed between these components sometimes it crosses this parent-child barrier And I said low complexity and this looks bad But each component only only communicates with the parent component. So you have a simple interface from the from each component but You can have multiple file system servers which would seem to make this really complicated But when we run Unix programs in our Unix emulator, we have this nukes process This is a confusing name but And so nukes sort of acts as the kernel And there's this VFS virtual file system layer in the nukes runtime, which is just like the kernel and We broke the VFS live we vote we broke the VFS layer into a library so You can use the VFS to communicate over an FS session With a block server And you could I don't have the MP implemented yet 9p implemented yet, but you can go over network So where do you put Nix? So When you configure a VFS either in the the nukes runtime or in a Program that has the VFS lab or LinkedIn Up here you see VFS root label That is a connection to a file system server and the root argument says change root as soon as I connect and the label is provided as a For the parent to decide where to route which file system server you're wanting and so I have nukes package written up here because These two these two top connections presumably I want them to go to the same place But my for my home directory. I want that to be a different server Um, so the trick is if you can rewrite these arguments at the parent So up above say we have this VFS configuration. I Don't send the session request to a to a file system server. I send it to this Nix server and So whenever this sees nukes package it looks in this file nukes package slash default nix Which can be a really simple function that takes the root argument and So here I'm just taking this root argument out of a set Which presumably I'm rewriting the root argument to be a Store root and you can see here the store root doesn't have a leading slash on it, which is it seems odd But then once once I have a connection if I carry through the Nix server I'm just going to pass a capability back up and then you have a direct connection, but I have a true root so I've rewritten the the the the true root I'm using this and so What you can do is you can have the VFS in a server that serves FS sessions And then you just make one evaluation of the VFS server then all clients would use that that Nix evaluation And I'd like to talk about how it works underneath, but I don't have time for that So I can give you a quick demo This is well, maybe I won't show you that but just for brevity Okay, so I'll start bash and the the runtime and you can see up here I'm not building the packages because I don't have all the expressions ready for that but I'm using connections to a Nix store and so Well, there's probably a lot of garbage in here, but like I have this I have I Can look inside and see I have some store paths, but I mean bash bash is not a similar bash is a real it's a real file and This this launcher I have isn't doesn't have any sort of Nix support built into it. This is just a okay I've just sort of done this FS Stuff and so I have a really stupid example here that I came up with All right, so I have this folder Nix So what I'm fired I There's nothing here, but if I try and run a non-existent Script generating foobar.sh. That's the wrong extension Okay, yeah, you're using DOS box, but you can generate files next and so over here you can see I don't know why it requests the file over and over. I don't know that much about DOS, but If I if I have lib z I can sort of access Nix solely using the file system by abstracting different parts of the file system And there's a lot of other details. I could go into I don't think I've time, but yeah, that's it We got time for one question When should we expect like a full-blown talk because that's pretty interesting. Um, I was afraid the wheels would fall off So I didn't want to request a time slot The full slot but Again, I need to sort of clean things up and get things finalized, but yeah, hello, it's loading It's my time already like counting because I Thought I'll have a laptop and I'm I'm booting Manjaro Linux throw tomatoes at me It's like this not Nix OS thing. I Have an excuse though like I have to build stuff for our slinox So this is sort of my development machine. So then Okay, does anybody know how to use extra under Yeah Yeah, right Okay Okay, yep What I want I wanted I wanted to okay Let me look I didn't even take time to change the desktop background. That's how they like care Okay, let's just now wait for chromium to boot up. Well, cool Yeah, right There's that thing. Yeah, I think so. Yeah. Oh Serum one. Oops. Oops. Oops. Oops. It didn't say anything. We'll go full screen Okay, cool. Yeah, I understand Yeah, that's that's that's a nice idea. Yeah, I mean it doesn't have text Oh How did how did that happen? Anyway, I Opened the terminal and then everything worked cool. So started timer the topic of The topic of this talk is quest for distributed Hydra and you might wonder like what What do I even mean with with distributed hydro? We already have built farms, right? so companies we build a lot of stuff and You know, we sometimes want to add some servers to our built cluster through Amazon EC for instance and Not do any configuration. Just you know, just have a built cluster like an Erlang and forget about it Also in ideal world we would laugh for You know people in next community to be able to contribute their computation power to actually You know offload the load from the main Hydra instance to build binary caches for the people and and here's the problem of trust Right. So we want so we we have two goals with this quest. We're on first goal is zero Configuration Hydra clusters and the second goal is zero trust Hydra clusters Okay So We we we do not want to you know go full-on and redo Hydra or something We want to take things very slow one step at a time and begin with figuring out how to actually use, you know distributed build PL script, I think it's called to to to implement some strategy of You know distributing the bills because some some strategy of distributing the bills other Than fanning out the tasks with really smart locking right basically now. We're as far as I understand. We're fanning out the tasks You know across the the machines that are in our Knicks dot build machines and Be and being really careful about how we lock stuff so What we want to do to begin with as to do to use consistent hashing To say okay, like we have this set of like in our company. We have this set of packages we're building and We will build, you know that packages on This like we will hash Package packages and we will figure out that at this state of cluster these machines are building that packages this way all our clients that want to get binary You know binary outputs will know from you know from consistent hashing algorithm from where they should obtain the binary the binary caches and Of course we we Have no idea how to solve the The trust problem. So this is why my colleague as Has chosen this as his master's thesis. So you have have if you have any like Guidance and your trust system systems then and if if you want to join us on this quest then you should drop us a line That's about it Okay, thank you Okay, I'm sure there must be questions about this Oh, so you you actually talked about two things. So Distributed hydra and then you were mentioning the binary caches, which is about the storage. Yes. Yes, I Maybe wasn't clear about it. So basically we want to experiment first with you know distributed build PL script Sort of in sort of integrate a consistent hashing there Right and see how it plays with the entire process of of of building stuff using the feature of nicks We should we just call distributed builds and then sort of taking it from there with the with the experience We got and like with the pitfalls we fell into Do the proper thing with hydra? But that's our vision of how we should do and we like by the way We don't have like any prototypes. I can't show any demos. It's just it's just that we think that eventually I Mean for for the community. I mean for companies we can write workarounds and whatever but for community It's extremely important to figure out this trust thing Right so that I can that so that I can in like one click donate my computational power to the community so that you know, we can we can build Packages for we can build binaries for the people that's that's our like and go Pete has a question I guess I'm curious about what you really mean by zero trust so because There are different levels of trust right you can say I have public keys of people Configured and then I trust those or do you have this web of trust kind of thing you have this Why I'm not not quite sure how to interpret this requirement. The thing is that we don't have You know any Vision of how it should be done But we have some ideas. For instance, we can implement something like notary systems when the In in our cluster and this is where consistent hashing and if you have noticed like this is basically a slide from Basha guys, right? So So we have we have more than one server building the same thing and then notaries are saying What is your output? What is your output? What is your if there is a consensus of outputs then we Sort of trusted and of course we have to make sure that in our hashing function Somehow we know that these three, you know servers are owned by different parties Right that are not that they didn't don't have a conspiracy So this is this is sort of like what we're thinking about when we when we're talking about zero trust Hydrofarms and some hints you might want into spoofing and civil attacks I'm sure civil attacks s y b i l attacks and spoofing Okay, what what what what what do we spoof? Hmm like What is the the threat because you may because you said you you have to make sure that there's not an alliance because spoofing is essentially When when more than one agent collaborates on Abusing the system the trust system So just some just some hints where you might want to look which which keywords you might I mean we can talk about it, but I mean I'm just stupid to see To see how it's immediately relevant like if we have good enough notary system then We just speak like random no, I'm sorry We pick notaries with trust I guess and I mean we'll talk about it later I'm just stupid to to answer that or think about it right now anything else So the next Presentation will be by Jack Cummings who will be talking about Hydro in more practical terms I would say I think so I wrote this Five slides during one of the presentations this morning. Someone was I was noting that Hydra is a very very useful tool if you're doing internal next deployments and It's actually kind of a bit of a work to get going This is my experience of setting up Hydra. So things have probably changed since then I think I tried repeating this with the Latest pull of Hydra and it didn't merge well with some changes I made one of the Problems I have is my builds can take hundreds of hours So it's really annoying when a bill takes 99 hours and the time out is 100 hours that you lose 101 hours of the work My Hydra server doesn't run on Nick sauce it runs on a SUSE server It's the next I'm using as a multi-user and it's installed that doesn't use Nick store as a Nick's path And it's all internal to the company work for so This is kind of a one of the less traveled paths than Nick's I think So the first thing you do is you need to grab Hydra That's kind of a good way to get it Then you build Hydra. It actually uses Nick's to build Hydra One of the less known facts is you can actually use Nick's as a replacement for make And more You also need Postgres there was a time that Hydra worked with SQLite But it does not work so anymore that one took me about two hours to figure out Then you make your Hydra database Create a user for it pick a password throw it into PG pass so that I see tech ate my tilde, but Then you can use locals authentication Static disclaimer when you're using by these things If you start up a postgres server on the big bad internet it might actually only yeah, who knows what that happens So the next step is run Hydra in it and that sets up all of the Loads the schema into the database sets up. I don't know a few other things that Hydra needs to get going and This is all stuff you run once the first time you set up Hydra and then every time you run Hydra I have just a little script to do this I do need to wrap in this in a script because I have a different store and state directory So a lot of the next stuff has Nick's store hard-coded into it, which I tend to discover on a regular basis Then once you've got The variable setup there are four parts that you need to run currently There is the postgres server which can contains all this state the state about all the builds and all the evaluations the qrunner which dispatches everything the evaluator which periodic the evaluates all of the Expressions you tell you can figure Hydra to deal with and then a server, which is actually the web front end Once again, I've never had an internet-facing Hydra server So who knows what kind of happens to it when you put in the internet? so Then as far as this goes, I believe the default is port 3000. I'm not sure that may have been semi-configured somewhere I can't remember This is actually the easy part once you the hard part I had with Hydra was Getting all the dependencies and get it running and once it's running the web interface is actually really easy to use just setting up inputs to a job set and Pointing it at the derivation you want to run It is worth noting it's worth backing up the Hydra database quite often because in the application we use it for which I'll talk more about tomorrow I Tend to use it for doing the reverse mapping of what derivations mapped back to what source sources, which is something that I Kind of miss a bit about next is being able to say oh, I have this derivation in a star now How did I build that or what did I what sources did I build that from? so Yeah, that was kind of a quick World round tour of the guts of setting up Hydra Anyone anyone Could you tell a bit more about that reverse mapping that sounds? How do you actually do that sounds interesting? Okay, so I Have Hydra set up to evaluate like every 60 seconds because I don't really mind burning up a bit of pure resources So when Hydra evaluates a branch and one of the inputs changes, it'll reevaluate it So what happens is in the input tab of Hydra? It'll keep track of what? revisions that used from what SCM repositories to build the derivation so that That turns to be the more and more useful features I use in Hydra is being able to look at the big table of what jobs succeeded at point find the green check marks in the mess there and find out what revisions that was and be able to Track it backwards from the derivations what changed in all the inputs. This is since I find it very useful because my jobs can some takes time to take hundreds of hours, so you have 10 or 15 in the pipeline and The figuring out when stage 13 the pipeline failed what version that mapped back to you and be able to figure out What you change to break everything So that's how I use that Okay, so for the final talk for today Will be by Bob long as you can already see on the slide and it's going to be about a Haskell web framework As far as I know Yeah Hey, everyone. My name is Bob. I'm from Dublin, Ireland. I'm over in Berlin for the week or so I just wanted to quickly talk about something that we've built a work using you so and we're building it with Nick's and deploying it with Nick's So I work at intercom What we do or kind of mission is to try and make web and mobile Business a little bit more the communications involved a little bit more personal So we've ever had a bad experience with a help desk which I tend to do multiple times a week We're trying to kind of solve that problem So we build a couple of products around customer communication But we also have a fairly good API and webhooks offering so you can build your own kind of software to do some of this stuff One of those kind of things that we offer webhooks so When we built webhooks first, we thought this was a cool feature and then a couple of months later We realized that people were building real businesses on it and we were not giving it the respect that it's deserved You we we obviously tested and monitored every single part of our webhook delivery pipeline But what we weren't doing was in an end-to-end way doing some sort of continuous quality assurance Which is difficult to do because if you think about what webhooks are webhooks are something that are fired Sometimes seconds are more after they are triggered and there's no request response model So this isn't like a typical web development testing tasks. There's a there's an asynchronous nature there The other problem is that often you're investigating or want to investigate an issue on a production system But you're on your laptop on the bus or something But in order to receive webhooks you need to have a web server running and if it's on a if you're subscribing to a public Production system, then you need a public IP or DNS to to subscribe to So we've been working on this tool called Shelduck. It's called Shelduck because the team I'm on an intercom Names all of our products after birds and we've been doing a lot of products and now the birds are getting weird, but We're on Shelduck right now And this lets you write simple API request webhook Expectations and then Shelduck just does all of the machinery behind it So this is what it looks like each of these blocks is an event that Shelduck has observed related to the webhooks pipeline So if you're trying to investigate an issue, this is where we go now And this tells us what's up. It's also got integrations with Slack So if you configure Slack, it will send Notifications on test failures and your colleagues will know you're writing Haskell because the F is in capital letters It's also got Keen.io support So we do analytics based on failures and it just sprays all of this stuff into these different services So it's got a lot of concurrent components. It's got a test runner or a quest engine is hitting the API It runs a separate webhook web server to receive those webhooks And then it runs an optional SSH tunnel so that you can test this on the bus And it runs the ASOD web server, which you just saw You saw it's cool If you like web development and you like Haskell and you like DSL's that's the Venn diagram Then you saw it is for you in Haskell. We have the notion of quasi quotes Well in GHC, we have the notion of quasi quotes where we can kind of embed these DSLs into Haskell You saw it itself is kind of I think they build it as kind of a traditional MVC framework with Haskell's type system used To go to fact it rules out a lot of the types of errors that web programs Web programmers are pretty used to it's got a bit of a reputation of being difficult Which I think is slightly unfounded if you look at the core Simple ESOD web app is about 20 lines to get a hello world thing going But if you're if you want to do web development in Haskell You should just pick a framework because if you don't like something generally things compose pretty well So you can just rip out parts and replace them with other parts So it's not a big decision you have to make which is nice. You can't do that with rails and Sinatra They don't compose at all So I feel a little bit embarrassed about the slide after listening to a day of cool stuff about NYX deployments But I thought I was really smart using NYX copyclosure to do this But my needs are pretty simple. We just have a fixed EC2 instance and we use NYX copyclosure to send this Closure over. I will probably be doing NYX ops in as a result Intercom is pretty crazy about deployments if any of you've read any of our blog posts We do like a hundred deployments a day about without any downtime And we've got a great ops team who care a lot about this stuff one advantage of NYX copyclosure is that it's so kind of Small it requires very little of an investment with a From your infrastructure team if you have one So if you're looking for a way to kind of sneak NYX in initially, maybe this is a good approach as opposed to a full NYX ops thing. So that's all I have Thank you Sorry Sure So when you're doing these builds to do NYX copyclosure is that do you have a Pre-configured machine that knows about how to run NYX or do you have all of your colleagues having to have them in store? Nick so they can do deployment. Yeah, so the way we we Do this is that we use virtual machines to develop it and It's like a vanilla NYX OS box that I think we install using vagrant or something because that's what everyone uses in work And so it's pretty easy to get up and running in about 20 minutes or so Just by following a cell list of commands and then you can start deploying to our fixed instance So you have multiple people doing that. Yeah a couple Are you using a servant? Oh? No, no, no, we we so what's interesting about? Servant is Kind of build as a very type safe DSL and Haskell for generating RESTful APIs, is that fair? Yeah, just Didn't pick it, but it does run as I said two web servers one is in your so for the main UI And then another one is just for receiving Webhooks so maybe that would be a good good pick there because I can swap that out pretty easily