 Who learned something really cool today? I think the RustConf organizers deserve a round of applause for that. All right, I'm Julia. I am on the internet. I work at Stripe. They helped get me here. That was wonderful. Cool. So I'm going to talk about a bunch of things in this talk. I really care a lot about learning systems programming. I'm going to talk about how Rust helps facilitate that. I'm going to talk about the particular ways in which I think the Rust community is wonderful. We're going to talk about contributing without coding. We've talked a lot about contributing with coding, which is also wonderful. And we're going to talk about learning all the time, which is my favorite thing. So there are a lot of reasons to love Rust. I hear that people love Rust because you can write fast and safe software. We've heard a lot about security. There are a lot of really great programming languages ideas. And I am not a Rust wizard. In fact, I live in a comet around Rust. And I spend a great deal of my time quite far away, being like, what is a borrow checker exactly? Like, I sort of know. So I'm sort of like a maybe beginner intermediate Rust programmer, right? And I do not write production Rust software at all. Instead, for me, Rust is a really good way to learn about systems, which is something I've spent a lot of time doing. It's a fun way to earn experiments. And it's a good place to go to ask for help. When I got invited to give this talk, which was the best day ever, I got an email of being like, so we like to think of Rust as being empowering for a wide variety of people who might not otherwise consider themselves systems programmers, right? And this is something which is really exciting to me. So Rust is really good for experiments. I like to do a lot of experiments. I have some rules around programming experiments, which is that they don't need to be very good. It can be like a little tiny program, which doesn't even work, right? Maybe it compiles. Maybe it probably compiles. But that's just kind of like a minimum. Like, maybe you probably want it to run. But the only important thing is that you learn something, right? And then maybe you can tell someone what you learned. And so a few years ago, three years ago, I was kind of in this place where I'd been working as a web developer. I'd been working as a data scientist. I knew a whole bunch of programming languages. And I've been programming for a while, like for maybe 10, 9, 10 years. I had two computers, nice degrees. And I do not know what to learn next, right? And I still felt like there was a lot I didn't know, which was true, right? Like, I was into systems programmer, definitely. I did a lot of math. And in math, you're like, what is a computer, right? So I didn't really understand what my kernel did, even though I'd been running Linux for a long time. I was confused about some basic facts. I didn't know what a system call was. I had a lot of really basic questions about how operating systems worked. At this point in time, I went to the RecurR Center, which is one of my favorite places in the world. For those of you who do not apply because you don't know what it is, the RecurR Center is a 12-week program, 12-week programming retreat, where you go and you're like, what do I love about programming and what am I going to learn? And you learn things on your own and you decide what's exciting to you. I went there with a lot of questions about operating systems because I was like, what's really hard? And I was like, building an operating system would be really hard, right? And so I would ask, like, what's a system call? Because I didn't know what that was. And then someone was like, well, it's the way you tell your operating system to do things. Like, when you open a file, your operating system knows how to open files. And then so you tell it with a system call. And I was like, oh, okay. Now I know what it is. Done. I wanted to write an operating system and I decided to do it in Rust. And in particular, an operating system is a really big thing. And I only had three weeks. So this was unlikely to work. So I focused on writing like a keyboard driver, right? Where you type the key on your keyboard, like J, and it appears on the screen. I used to have a demo of this, but my Rust code has not compiled for the last three years. Because I wrote it in Rust 2013. Anyway, it's a whole thing. So I wanted to write a keyboard driver. And there were a lot of questions involved. Like I needed to use assembly, and I needed to use strings in Rust. And I was really confused about strings in Rust. And two kind of delightful things happened. One thing was that I would go to the Rust IRC channel and people would answer my Rust questions. And then sometimes they'd be like, why are you doing this? I'd be like, well, I'm trying to do this thing with assembly. And then they would also answer my questions about unrelated things, like assembly, which was extremely helpful because I did not know the answers to those questions. And I wasn't quite sure where to get them. And we're gonna come back to that. And I discovered this really interesting thing, which is that even if you're a relative beginner to a thing, like I was to Rust and to operating systems, you can somehow still contribute information to your community. So I was writing this blog in 2013, because I was at the RecurR Center and I had like, I didn't do anything else with my time. So every night I would sit down and write a blog post, which is like, what is ELF? ELF is the binary executable format on Linux. And I was like, what is it and like, how does it work? And I was like, I tried to write a keyboard driver and yesterday it crashed and the day before it crashed and the day before it crashed, but today it didn't crash. And I was so happy. I wrote like some BuzzFeed style articles, like 12 things I learned about linkers. And it turned out that not, in fact, like I didn't know this stuff and it turned out that other people also did not know any of this stuff, right? And I could like contribute something even though I was kind of a beginner. And there's this really interesting comment in the opening talk this morning. I think by Nego, which is like, if it's not documented, it might as well not exist, right? So there's this idea that like, if you don't know about something, it might as well not exist, right? In a way. And so this is how I became a chief developer advocate for S-Trace because one day someone told me that a program called S-Trace had traded system calls. And this was so like, it was so upsetting to me that I hadn't known that this existed for the last 10 years that I kind of embarked on a campaign to tell everyone I knew. Currently I have printed 500 stickers and I have a S-Trace sticker on my phone. Anyway, it got sort of out of hand. But the reason I kind of like doing this is like that I program in my job, right? So I spend all day like doing code reviews and programming and like, you know, writing design documents. And at the end of the day I'm like, oh, I don't want to do another code review. It doesn't sound fun, right? Or it's like too much to say. But I do like to tell people about things. And the kind of fun thing about telling people about things is if you're building something, then you have to like build the whole thing and then tell people about it. But you can do this cool trick where you just skip the building it part and find something really cool and tell people about it. And then people are like, wow, you're amazing. You're like, I know, right? I just invented S-Trace for you because you didn't know about it before. So it's basically like I wrote it. Okay, yeah, so that's it. That's my pitch for telling people about things as a cool hack to become the best programmer in the world. But if you actually want to write programs and learn systems programming, how do you do that? Right? And so I have this coworker who asked me yesterday. He was like, hey, I'm reading this Rust book. I've been learning that syntax. I'm like, but how do I, like what's a good systems programming program to write? And I was like, good question, right? Like you want to learn something. You don't really necessarily mind what you learn. You just want to be new. So we're going to talk about another experiment, which is one evening at Julia's house. And Julia is like, okay, so like what's going on with concurrency, right? And so let's make this a little more concrete than like what's concurrency? And there's this question of like, what are the systems primitives for concurrency, right? Like I have a computer, they're concurrent programs and they're all kind of using the same stuff. And so I was like, okay, I know there's threads. I know sometimes I was really into S-Trace, so I would S-Trace things and I would say few texts. It wasn't totally clear on what few texts was, but I think it has something to do with concurrency, right? And I knew that you had like atomic something, I don't know, right? So I decided to do some programs. Step one was I wrote a C program because ways you're to write unsafe C programs and unsafe Rust programs. Cool feature of C. So I made like a thousand threads and then all the threads would increment the same counter, right? Which is like how you get a race condition. And I'm like, I made a race condition. I'm so happy, extremely easy. And I would run it and it would not get to a million. It would get to some lower number, right? And it was always a different number and it was very exciting. And I felt proud of myself. And I'd already succeeded, right? I made a race condition. Good job. And then I was like, okay, I'm gonna learn to mutex this, right? And so I wrote, I figured out how to do it in C, which is use this function called like P thread mutex lock, I guess. And then that maybe uses the Futex system call. I showed this talk to someone before I gave it and he was like, did you know that it only sometimes uses the Futex system call? And also sometimes it'll just pull and I was like, oh, it's so complicated. Such a world of excitement inside this little tiny problem of you're just like trying to increment the counter. So I did this, I used a mutex where you're just like, okay, I'm updating this counter. I'm gonna like let me know when I'm allowed. And then you update it and you're like, okay, I'm done. And then someone else does it after. And that worked and that was great. And I felt like I knew what a mutex was a little bit, which was good. And I also felt like I knew what system calls were happening, which is good. And then I was like, okay, so this is atomic thing. What's up with that? So I Googled, I was like atomic computer, I don't know, not atomic computer, but something smart. And what I figured out on that day was that you have these atomic instructions for x86, right? Where you're like, please increment this, but in a smart good way where you put a lock around it. But in just like one CPU instruction. And then this actually explained like, so I got in like somewhat earlier, I think also in 2013, I was like looking at this job and someone emailed me this list of questions and they were like, can you discuss like the pros and cons of implementing a lock-free hash table? And this was kind of interesting because at the time I had no idea what this meant. And no idea how to discuss the pros and cons. And I was like, this seems very interesting, but I'm really confused. And then so when I learned about these atomic instructions, I was like, oh, I would like use atomic instructions, I see. And then like, I still don't know what the pros and cons of implementing like a lock-free hash table, and that's okay. Maybe one day I'll learn that. Okay, so this leaves us with kind of like a fun exercise, right? You could like write a multi-threaded program where all the threads increment the same counter and you could do it two different ways. And then maybe you'll know something more about systems programming than you did when you started. Okay, so that's it for experiments and like fun things. And like these are all nice because they're all sort of like unproductive in one way, right? Like you're not producing new wonderful code that's going to change the world. But they're producing a lot of like knowledge in yourself and maybe in other people, which is really nice. I want to talk a little bit about actually writing programs because I read a program in Rust recently, maybe for the first time actually, like a program that does something useful that had previously been a very low priority for me. And so I want to talk about this idea that like Rust makes kind of improbable programs possible. I originally wrote impossible, but I'm really, so like you can write any program you want and see, right, in principle. But it is somewhat improbable that I would write a complicated program and see for reasons that are going to become apparent shortly. So it's sort of important to me that Rust exists, right? And this is where we get back to this idea that like Rust is empowering, right? And you're like, oh, maybe I can write that program, right? And empowerment is just like about being able to do stuff. It's like maybe you can do stuff with Rust that like you wouldn't really do otherwise. So here's like the kind of problem that I was interested in. It's Ruby is taking it all your CPU and you're like you're ruining my day, Ruby. Why are you doing that, right? And you wanna know why? And I was a little mad about this because you can see that Chrome is doing actually. So let me start a video in Chrome really quickly. I'm not gonna put it on the sound, well, I could. No, anyway, you might know this video. And if I run a perf top, let me just, okay, so then you can see there are some functions in Chrome that are happening. This is what's happening right now on my computer. And there's this function like scale YUV to RGB32 row SCC. I don't know. So I don't know what this is, but I do know that it's taking up 3.59% of my CPU right now. And that's like some internal function in Chrome. And it was kind of upsetting to me that I could learn this about Google Chrome in like approximately three seconds. You can also see p-thread and mutex locked down there. It's a real function that's happening. 0.53%. So yeah, it was kind of upsetting to me that I could do this with Chrome, but not with Ruby, right? Because like, why not? That was mad. So what I wanted to do was write like kind of like a Ruby top thing, right? Which was telling me what Ruby was doing. That's supposed to say PID, but it looks like PIP. I was supposed to fix that, and I didn't. Anyway. So yeah, let's say you want to do this. And I'm really, I don't know anything about Ruby internals, and I was really scared of them. So I wanted to do something which was like outside of the Ruby process. And so the idea is like you kind of have some running Ruby process, and you just spy on it from the outside, and you want to see what it's doing. And so basically what happened is I thought this was kind of, I was talking to someone about this, and they were like, Julia, you can totally do this, it's really easy. Like just go do it over a weekend. And I was like, I'm not sure I can do this. But I decided to try, because this guy I was talking to claimed to was totally easy. And so the situation that he kind of impressed on me over many weeks, because you'd be like, it's easy. And he's like, have you done it yet? I'm like, no. I don't know what you're talking about. And so you kind of have this picture, right? And so you can see over there, I get the C stack in the Ruby virtual machines address space. There's its regular stack, which has unhelpful functions like VM exec or something, which is just like unrunning a Ruby function. You're like, great, really helpful. And then somewhere else you have the Ruby stack, right? Which has, I don't know, functions like Rails, something. I don't know Rails. I just assume all those functions are called Rails. And then maybe there's like extra information, like function names that's scattered all over the place. But in there, like inside the virtual machines process somewhere, there's information about the Ruby stack, right? Like it's not impossible to know what it's doing. And so I kind of like learned, I had this GDB script and it translated it into a program that didn't use features. Anyway, I'm not gonna go into this. But I wrote a C demo. And so I know C sort of, I know how to allocate memory in C. Technically, I know that you can pre-memory in C and I have called pre-before. But memory management in C and C is not a skill I have, right? And I didn't really, I wasn't really excited about learning it. So I sort of know Rust, but I'm not like super good at it. So I talked to my partner and I was like, well, I have this C demo and he was like, well, let's just translate it to Rust. And I was like, yeah, let's just translate it to Rust. So he translated it to Rust and took like a couple hours, right? And this kind of helps with my productivity. I have this highly scientific metric of program workingness. So it kind of went up and then I said to Rust and then I was like, oh, I see there are hash tables in Rust. I don't need to like write those from scratch, right? Or whatever. I don't know what C programmers do. I just don't know, right? Like I wouldn't know what to do. I could look it up, right? Like it's not impossible to learn C. I have not done it. So this is pretty helpful. But then I run into a second problem, which is like, if you have a process, you have a bunch of memory in your process, right? It's just like zero, one, zero, zero, right? There's some bytes and stuff. And you might want to know what those bytes mean, right? If you're like spying on the process. And in particular, there might be some struct, right? Which has like a pointer length. And if you know about compilers, which you, I don't know, compilers like will throw away a lot of this information, right? About like what the structs in your program are and they'll do like, I don't know, and they'll just like do a lot of pointer arithmetic. So you won't know if you're looking at this like zero, one, zero, zero, one. And you think it's like an RB string T. You won't know like what all those bytes mean, right? So what you want to do is you want to have like the original struct definition. So you can say like, okay, the first four bytes are a pointer, the next four bytes are a length, and then the next 16 bytes are a weird thing. So I needed to do this. And it turns out there's a debugging format called dwarf, which has this, which is, it's a whole thing. So I started out using this library called LiveDwarf. And I was like, I don't know what's going on. This API is really difficult to use. It was also slow, and I had to link it. And it was like a whole, it was kind of a mess. And I spent like 500 lines of like rust wrappers for LiveDwarf functions. And I was kind of having a bad day. I think I stopped working on this for like three weeks because I didn't know what to do. And I would kind of complain to people and they'd be like, yeah, that sucks. But like, so one thing that I do when I have problems is I complain about them on Twitter. And someone was like, hey, this is Rust library called Gimli. And one of the people who works on the library, Nick Fitzgerald, who I got to meet today, was like, hey, yeah, it's like not done, but maybe it'll help you. And like tell me if I can help. And I was like, oh, sweet. This is way better than LiveDwarf, which is just like this like software from the 90s or something, but like, I don't know what to do with. Maybe it's not, I don't know where it's from. Anyway. So, Gimli had less features, but it was easier to read. I could like go look at it. And they have like friendly maintainers and I would be like, I have a question. They'll be like, I will help explain to you how Dwarf works. Which was really nice. And I submitted like a three line pull request. Which is my like open source contribution for the year. And so the key thing here was actually that this let me get my program to work. Which is something I could not do before. Like I got like 90% working with LiveDwarf, but 90% it means nothing to your computer. So I actually got my program to work. It worked on three different computers. And I started to kind of understand how Dwarf worked, which was really nice, right? Cause it'd be like understanding systems and systems programming. And I was like, oh, I get it. Like there are these different, so in a program you have these different ELF sections. So you have code and you have code and you have data. And then Dwarf like put some extra debug sections at the end, which are just like extra train cars, really. And then there's this like, I mean that's like only like step zero of knowing how it works. You have to also know what's in all those sections and what it all means. And it's a whole, I was talking to someone who worked on GDV and he was like, it's a whole saga, but anyway. And then so we did it. And I should have this to work. Which was very exciting. I can show you sort of a demo. So this is my Jekyll, my blog building in a loop, right? And then, oh no, that's not it. 7088, is that it? Yeah, yeah. So these are like the functions that are running right now. I made like kind of a list of, maybe I need to make this smaller. Oh yeah, okay, cool. Yeah, so it's doing parse, parse seems to be really important. Sanitized path seems to be important. I don't know. We can kind of see like a top list of like the functions that are running. And it's all like real. And so I felt kind of happy with myself. I learned that desktop software is really hard. I mostly work on servers. It runs on my computer and also two other people's computers. I have no knowledge that it works on more than three. And so like, I often find rest kind of frustrating because I'm like, oh, there's a compiler and like what does this compiler mean? But like ultimately it helps me do kind of awesome things, right? And I can write some programs that I couldn't really write before, which is really exciting to me. I just am closing like a whole bunch of statements. I guess I'm really into like learning all the time as you may have noticed. And like one really fun thing about systems is like there's always more stuff to learn, right? Like I keep on meeting people who like, yeah, I've been working with Linux for the last like 15 years and like I'd learned a really cool new thing yesterday. And I'm like, great. I'm in no danger of knowing everything. And I find it really fun to like contribute to the community by like blogging and writing things. And like this is really cool, right? Cause it can be a lot faster than contributing by writing code if you like writing. I often will like write a blog post in like a couple of hours. So I'll be like, I learned a cool thing about S-Trace. Here's the cool thing. Goodbye. Right? Yeah. And another, one thing about blogging that I'm really into is making resources for like kind of intermediate developers. I used to find it frustrating when like I would see a lot of really cool tutorials about learning how to program, like learn Python the hard way. And like those are really important. But at the same time, I'm like, I know how to program. What I want to know is more things. And I think it's really fun to write things. Like you're like, oh, this is how a database works. And you're, you could be like, oh, I've been using databases for six years, but I never knew how it works, right? And there's like, there's always like so much more to know even if you've been learning, even if you've been knowing how to program for a long time. And one cool thing that you can do when you're doing this is like make things sound exactly as hard as they are. I think a really good example of this is computer networking. When I started learning about networking, I was like, networking is hard. It's confusing. I don't know what's happening. What I realized is like, now that I know some things about networking, I'm like, well, like with an IP packet, you have an IP address and a port. And if you know there's an IP address and a port on a packet, you already know a lot. And you could explain that to like almost anyone, right? Or like, and like maybe you need to explain what a port is. But like a lot of the concepts are like, not that unforeseeable. And so I wrote this zine called like Linux 2.5 and tools you'll love, which about, and especially, and included a bunch of stuff about networking. And someone sent this kind of like, tweeted this really surprising thing at me, where they're like, I've been using it to teach my eight-year-old. And I'm like, I don't think that you're teaching your eight-year-old about TCP, you know. I'm like, are you sure? But it's really like charming and delightful to me that people are like, oh yeah, anyone could learn this stuff. My eight-year-old could learn this stuff. And I'm like, whoa, well, like I'm glad, right? Like it's good. And sometimes like more people can learn more, like people can learn harder things than you think they can if you just explain it in a way that makes sense, which is really cool. And it means that we have to like, maybe get rid of our conceptions of ourselves as wizards, but maybe not, because I have a wizard hat right there, right? And I think that the REST community is a really good place to do this, because I've been meeting all these people today who know all these amazing things, which are not necessarily related to REST, right? But they're all here together. And I'm like, oh, you know all about debugging tools, or you know all about Linux, right? And it's really delightful to me to have all this knowledge about things that are not necessarily REST, but that are important to each other, like important and related to each other, kind of in the same place. And I think that's really cool. And so what I really want to close with is that I think it's really nice to have a place where you can kind of walk in and you can be like, what's a system call? And then you can walk out, like many questions later and be like, oh, I can do systems programming now. It wasn't that hard. I'm a wizard, and so I'm a fan of the REST community. Please, keeping awesome. That's all.