 Yeah, it's okay, no worries. Can everybody hear me okay? Can anyone not hear me? Okay. Sorry about the confusion there. There's some problem with hardware, which is something we're not used to dealing with in the post-cloud world. So I'm here today to talk about how at SmartB, we've been incrementally adopting Habitat, which is a tool for building predictable perfect runtime environments for applications. If you were here for the crazy container debugging story that was earlier this morning, this is a product that fixes that problem. So we're going to talk about how that works at SmartB. My name is Blake. I'm one of the world's last remaining operations engineers, and I work at a company here in Berlin called SmartB. These are my various handles, if you want to look me up online. We are hiring. I'm required to say that by my employers. You can talk to me about that later if you're interested in what we're doing. We make a software service energy analysis tool. So we basically take electrical consumption data out of buildings, and we analyze it and we help consumers and big enterprises figure out how to basically consume less electricity. So it's a sustainability company essentially. So going back a whole bunch of time to why we did Habitat. So we also have a very tangible dependency management problem in this talk because Jamie, who's the lead engineer for Habitat, is going to talk about what Habitat is later today, like after the break, and I'm going to talk about implementing Habitat. So a lot of this is going to be like, what is going on? This makes no sense. I've never seen any of these commands before. So hopefully you just like hold all your questions, talk to me afterwards, talk to some of the other Habitat folks who are here, or talk to Jamie after his talk, and go to his talk for more getting started information. So I started the web operations and my Cloud operations, part of my career in 2009, worked for a startup back in the States. If you can't tell already, I'm American. I wrote my own configuration management tool at that startup. That was awful. I would don't ever do that, never ever ever write your own configuration management tool. Like you somebody else's, there's many better ways to do this. But one of the questions that happened to, that we were asking ourselves a lot at the time because we were doing, I think in nine months at that company, I did four 36-hour shifts. When I say 36 hours, I mean not leaving the desk like toilet breaks and that's it. Like no eating, just coffee, 36 hours, four times in nine months. Because we were constantly fighting our infrastructure, fighting our applications, and just suffering. So these were the kinds of questions that we started to ask ourselves as we worked our way out of that whole. We were spending a lot of time manually or somewhat programmatically configuring applications. We couldn't get reproducible deploys. We wanted to encapsulate the application when we deployed it, and we especially wanted emerging configuration. So when the state of our infrastructure changed, remember this is 2009, so we didn't have Cloud Native anything at the time. We wanted configurations that would just change as the infrastructure environment changed, but we didn't have any of that stuff. So we talked about it, but basically there just wasn't a tool to do this for us. What came after that was me doing traditional configuration management with like Chef and Ansible and Terraform for like seven years. Just like all the normal stuff that you would imagine doing for a config management, which is okay. But that has its own set of problems that go with it. We still a lot of the problems that we were a lot of the things that I was dreaming about having here weren't solved by configuration management. Mostly what we got with config management was everything's in code. I have stuff in source control. I can at least have a software development process to describe what I'm doing. But we still didn't have these big promises that we wanted from the dreaming thinking that we've been doing earlier. Then in the summer of last year, I saw Adam Jacob do it. I watched the video of Adam Jacob announcing habitat in Seattle, I think the Seattle office for Chef software. I've been using Chef for a long time. I've been using it long enough that my honeymoon period with Chef was long over. I wasn't in love with Chef anymore. I knew all of its idiosyncrasies and quirks. I knew how to use it. I knew what didn't work. When Habitat was announced, I realized that this was exactly the thing that we had been dreaming about back in 2009. These are the reasons that we started looking at adopting the tool. In Habitat Universe, everything is an artifact. Everything is essentially a tar ball with metadata. The metadata describes how the things that you've tarred up are going to communicate with each other and configure themselves and configure each other. There are very few abstraction layers in Habitat. There's a little tiny bit in the core that is Rust that does the really important stuff. Then everything outside of that is whatever language you happen to need for the purposes of the thing that you're packaging. A lot of the basic configuration is Shell, which means that you can go to basically any engineer who's had any kind of compute experience, and they'll be somewhat familiar. They can copy-paste, maybe not copy-paste, but easily adopt things that they find on Stack Overflow into the Habitat Universe. Also, there really isn't anything that Habitat does. It isn't an established Linux or Unix convention, so most of what's happening under the hood is stuff with dynamic linking or pathing, and a directory structure that you can just examine. So a lot of the black box magic that happens with something like Docker is mitigated here, or at least you can see what's happening much more easily. It's portable. When I say portable, it can run on any Linux essentially. So if you have Linux kernel 2.4 or later, something like, okay, yeah, the Habitat guys are like, yeah, yeah. So any modern-ish Linux kernel will run it, and there's also Windows support that is not quite as mature as the Linux support, but it's evolving rapidly. I think I hope that the community will port to other kernels in the future, but at the moment it's a Windows or Linux thing. But most importantly, the application is in control. So whatever that thing is that you're tearing up has complete control over its environment. So you're essentially, I remember working with a vendor that we work with at SmartB that makes our monitoring software, and I wanted to port their data proxy stack, which we use behind our firewall. I wanted to port that to Habitat and that involved building eight different things, well, more like 18 different things that we mostly see. I was showing them what I was doing and they were like, I think you become a distribution maintainer. I was like, yeah, that's actually completely true. You essentially have a custom distribution that you're running for each application that just has only what you need for that application. That might sound a little crazy, but it turns out that these are actually quite small packages once you've got them built. So we started doing Habitat adoption in the early autumn of last year, and we are do almost everything that we do now. Everything that we do in production and development is Habitat wrapped with I think one service exception and that's something that's coming soon. So I want to go through some of the basic things that we did and how we did this incrementally. So the first thing that we did is we use Habitat as a package manager. I can tell you for sure that Habitat is now my preferred package manager. I don't package anything with anything other than Habitat, now that I've gotten this far. Like this is it. Previously probably package source is my favorite because it was quite portable. But this is much easier to control than app to yum or any of the older or any of the previous packaging systems that I used. And we'll also talk a little bit about like how we use library packages where we essentially just get like a binary or a set of libraries from a package we depend on. Because Habitat does dependency management as well. So this is how we, this is what the command line interface or like the CLI API looks like for this. I'm being told that I have five minutes left. So I'm gonna go very fast because this was a 45 minute talk. So this is how we do package management. So you do like have package install is the thing that you want. And if you've been like it, you get something in your path that you can call. Or you can call it in sort of like bundler style like have package exec. And you can call like directly to the package and call your binary. This is how we do referencing for another package. So I think what's the, Fletcher, do you remember what the model was that you guys use for packaging to arch? It's kind of like arch, yeah, so we borrowed very heavily from arch. I think for like the way that the package dependency stuff works. So for Habitat, also this process supervision, so and I want to talk very quickly about how we do orchestration and bootstrapping. So we bootstrap the whole blob with a system D service, right? And that starts the supervisor and then supervisor manages the actual Habitat processes. If you know Terraform, this is how we do a, this is how we like provision a Habitat service on something we're terraforming. So we copy like a Toml file, which is like a configuration format. Copy that onto the machine and then we load that into the service. So you start the service, the service isn't configured, it's crashing a bunch, but it just restarts because the supervisor restarts it, then we inject that configuration. There's also now, as of like a week ago, roughly, there's support for private origin, so you can, there's like a build service that the Habitat team runs that you can get your packages out of, that's like your repo. And we now have support for like private repos that you access with Git Habats so that you don't have to put like private code out in public. There's a couple of things you need to remember if you are, if you decide to try using Habitat. One of the big ones is you have to love CI slash CD, right? Like if you're not doing CI CD, you're not going to be happy with Habitat. That's what it assumes. It assumes that you have that kind of change control, automated change control. You do also need to accept that like there's churn in the core and there's churn in plans, which is like the package definitions, because there's, this is like a very rapidly evolving project right now. We're still, we still haven't hit version one. That said, we're running all of our stuff in production as Habitat packages, so it's definitely stable enough to use if you are aware of what you're doing. And a couple other things about this. So this is one of my favorite things about it, it's, it is very container friendly, right? So like the runtime is kind of immutable, but configurations can change if they need to, which is something that I as an operator really love. I also have sort of adopted the mindset that my package is essentially a binary. So I should, I should be able to like give some input to it and get some predictable output out. And that's really how we do our CI. Like we really focus on this sort of like, you know, does it work? Is it working or broken mindset? Not like it's kind of okay and I'm going to configure it after it's installed. And this is the most important thing I think that we realize when we're talking about incrementally adopting Habitat. Initially the increments that we were doing were like software development increments, but really what we should have done in a lot of ways is just rip everything out, like keep the test framework we had for the service that we had in our old configuration management, rip everything out, test only at the service boundary and then go service by service and just delineate at the boundary. And that would have, I think, made our adoption process a lot smoother. And what we've been learning really is that we really, that we need to trust the artifact. We have to, we have to take, we have to believe that promise that whatever we ran locally is going to run in production, which has really proven to be the case. A couple wish list things I'd like to see that we're actually working on, working on with the Habitat team. I'm not sure if I still think that I want Chrome like stuff. This talk's also a little bit old. A lot has changed in the last week. I also want, we do some of our testing in the package builds. And we're working right now on using the Habitat Studio, which is like the build environment as a development environment as well. And this is the thing that I kind of didn't explain because we didn't have enough time, but I'd love to talk about more is that I think Habitat is the beginning of what I would call like a movement towards open services. So not just having like source code that's open, but actually having the operational part of an application or a service also be open source. So the whole ball of wax, including the operational bits can get kind of like shrink wrapped and put out in the public eye and shared between operators. That's it. So if you have questions, I think you're gonna have to ask me like in the hallway track, there are a couple other folks from Habitat here. Can you guys stand up for a second? Just so people can see, sorry, but just so people can see who are. So there are a bunch of Habitat people here. So if you have questions about what the hell I just talked about, these are the guys Jamie's talking in like an hour, roughly, I think something like that. Jamie's talking about like getting started with Habitat, which might also be a good thing to do to go see after I've confused everyone. And that's it. Thanks.