 OK, yn berthynas i'n 130. Felly, rydw i'n dechrau. OK, ddwy'n dechrau. Felly, rydw i yw... Mae yma. E'n Jon Billings, rydw i gyllidio'n gweithio'n gwyllgor o'r 5 yma, ac, ydych chi'n gynhyrchu, dweud y gweithio'n gweithio'n gweithio yma o gweithio architecturau, o'r gweithio ar y monolith i'r gweithio'n gweithio'n gweithio. sydd o'n dweud y teimlo cyfnod, a dyweddyn swagger wedi'u meddwl gweithio'r proses. So, rydym yn ddu hwnnw, ydych chi am gweithio'n cyfrifoedd cyfeirio'r blodiadau diolchol i'r cyfrifoedd o'r cyfrifoedd, ac yn hynny'n cyfrifoedd. I hynny'n cyfrifoedd C3 2015, mae gennym yma 89 miliwn, yn ychydig o'r hunain gwirioneddau sydd yn blygu, ac yn ymwneud y 90 miliwn cyfnodd yn ymwneud y cwmpertyn. A oes 71% o'r cyfnodd yn ymwneud yn ymwneud yn ymwneud, wedi bod yn y web i'r app. A mae'n ymwneud yn 32 o'r cyfnodd. Felly, Ahttp Jason yn ymwneud. Mae'r unicorn, mae'r unicorn fath. Mae'r unicorn yn ymwneud yn ymwneud yn ymwneud. If you're in Java Land, then there are frameworks like DropWizard. Or if you are in Python Land, so I'm familiar with Python Java. There's HTTP Lib. There's SimpleJSON for doing JSON decoding. You've got frameworks like Pyramid. On the infrastructure level, you've got things like Vanish to do your HTTP caching. You've got ha.proxy and nginx to do load balancing. Ac ydych chi'n ddwyliadu'n hyn ar gyfer y Lai 7, a'r ddiwedd i'r hyn sy'n gweithio ar gymfio Yhtt. Rwy'n amdulygu'r hwn wedi'i hyn, am ddiwedd i'r ystafell, ac mae'r ddweudio'r bod yn gweithio'r ddweudio'n gweithio'r ddechrau a'r ddweudio'r ddweudio'u ddweudio. Dyna'r ysgol gŷt, mae'r ddweudio'r ddweudio. Felly dyna'r unicorn yn olygweithio'r ddweudio'n gweithio. Why is this? Why is the unicorn so sad when everything is so great? Well, here is a mythical situation that has never happened to any of us. The website is down again. Why is that? Oh, I just pushed the pet store service. So the pet resource, it takes a string, right? I pass it a string. No, it's an integer. There's confusion over this API. You know, curse you in your strings. So if only there are a better way to specify our API. So we do actually have a few options. I'm going to walk through some of them. So you can write specification documents. And here's an example of a spec doc for the pet store service. And just in case you don't know, the pet store service is kind of, is an example crud web app where you can kind of post new pets. You can search for pets. You can kind of download pet information. And so this could be an end point for the pet store app. And you see there are some attributes and some required or optional. And some descriptions over what each one of those means. And then there's a response, an array of pet objects defined below. So pros and cons. So it's really easy to get started with this. And this is what we did at the beginning when we were breaking out our services. And if you use something like Google Docs with commenting, then people can just look at browse those during review time and leave comments. And you can very quickly iterate. And if you're not completely technical, so you're a PM, then you can also look at those and try and sort of figure out what's happening. But it's not all great because certainly the implementation and the specification can drift over time. There's nothing actually keeping them in sync. Maybe at the beginning they are in sync, but then some people add some new end points, change some parameters. And after a few months, the spec doc really doesn't reflect reality at all. And it's also really easy to be imprecise because this is just written pros. There's nothing forcing you to actually specify how things are. So another option is you could use an IDL, an interface definition language like thrift or protocol buffers or Avro. And here's an example of a very simple multiplication service defined in thrift. And you see it looks very IDL-y. There's a multiply function. It takes two ints, returns another int. So what's good about this? Well, it can be very efficient on the wire. Because you're using an IDL behind the scenes, you really can really compress things on the wire. And it's really efficient to decode. So we certainly have situations where you send, I don't know, 10 megs of JSON, and it takes Python services, 100 milliseconds to decode that. And there's not much we can do to speed that up. Whereas if we use thrift, I think it's probably going to be more efficient doing that. But there are some downsides. So you cannot use nice layer 7 technologies such as HTTP caching. That's out, we're just at TCP now. It's difficult but not impossible to debug on the wire. And the quality of support for some of these IDLs can be a bit variable. It really does depend, especially if you're using a less well supported language. Another option is to write lots of integration tests between your clients and your servers. And then effectively the tests actually become the specification. You're basically saying something like, as a client, if I send this request to the service, then I should get back this response. So what's good about this? So you should hopefully be doing a little bit of integration testing already so that you learn about errors before you hit production. But it's not all good because integration testing is pretty much at the tip of the testing pyramid. You hopefully just have a few integration tests and they can take a long time to run because you may have to spin up some databases, spin up some services. It all takes time to do this. So the iteration speed is very slow. So ideally you want to find bugs before this point. Another alternative is you can write client libraries. So you hand your clients just a library in whatever language they're using and they just talk to that library and then behind the scenes the library actually does all the communication on the wire. So what's good? It's a black box of the client lib. Consumers really don't need to worry about what's happening. And from our own experience at Yelp, we have actually done this and it can work. But there are still disadvantages here. If you're writing a client lib, then there's lots of boilerplate, sort of code that doesn't really add anything. You just need to put it in there. And you're probably going to be doing manual validation, checking the strings of strings and the ranges are okay, that sort of thing. And you still haven't got the specification for the wire protocol. So you're not actually forced to write that down at all. And so you probably also still need some integration tests as well between the client lib and the service. Okay, or is there another way? So we could stick with our existing HTTP and JSON infrastructure. We could then just maybe, maybe we could invent a machine-readable specification language to declaratively specify what our endpoints or return types look like. Maybe you can see how I'm going with this. And then we could create a whole bunch of tooling to actually generate client libraries from these specifications of our endpoints. And then we could also create some tooling to perform server-side validation against these specifications. So are people sending us the right things? Are we returning the right things back from our server? And then maybe we could also create a really vibrant open-source community around all of this. But what am I talking about? So I'm talking about Swagger here. And there is a URL there if you want to look at the Swagger spec. And now what I'm going to do is I'm just going to walk you through Swagger. So the history of Swagger. It's been around since around 2011. That's when version one was released. And there have been a few different versions since then. I think in version 1.2 they actually formally specified the Swagger specification language. And then version two, which was a moderately big change, was released in 2014. And that kind of got rid of a few of the pain points. And then just this year it was renamed to the open API specification. And it's supported by Google, Microsoft, IBM and a few other companies now. So here is the pet store API. And you can actually try out hitting this end point. So if you curl it and you specify an ID, so 42 of a pet, and then pipe it through JQ just to format it nicely, then you get back this response. It tells you the ID of the pet in the category, the name of the pet, some URLs, and some tags and status. So this, for our services at Yelp, we do things that look a little bit similar to this. This isn't completely strange as a response from a service. And there's quite a lot of structure there. And as a result, it's quite easy to get something wrong when you're dealing with objects like this. So what would a Swagger specification look like for this? So there's actually a full spec at the URL at the bottom of the slide if you want to take a look at it. But what does the Swagger spec say? Well, it says it's version 2.0 and there's a description and an opaque version string and a title. And then the important things are these path objects and definition objects. That's basically where the meat of it happens. So I'm going to talk about path objects and definition objects in turn. So a path object for the endpoint that we just saw. So you see there's a path, slash pet, and then this path parameter, which is pet ID. And if you look down, you can see this pet ID path parameter is bound in the parameter section, name pet ID, and it's in the path, and then there's this description. We see it's required. What type is it? Well, it's an integer and there's this format, which is in 64. And I'm going to talk a little bit about types and formats later, what choices you have there. And then there's this response type at the bottom. And you can specify response types for both success cases and also error cases. I'm just showing a success case here. So if you get a 200, it's a successful operation, and we return this pet object, which lives in the definitions section. And so this is just a reference. And the nice thing about swagger is you can actually split these definitions out across different files, which can help keep it dry. So here's an example of another parameter object. It's not used in the example I just gave you, it's used in the find by status endpoint. So what is this? Now we're saying it's a query parameter, and the name is status, and it's required, and it's an array, and the array contains strings, and these strings range over available, pending, or sold, and default is available. And so this is pretty powerful. We've got an enum type here, and we're actually going to be able to check that people are passing in the right enum values. So let's continue with our getPet by ID. So this response type, which is here on the right, I've given you the spec for it on the left, and so you can see that there are a couple of required fields, and then there are a whole bunch of kind of required and optional fields defined below. So there's ID, there's category, there's name, there's photo URLs, tags, status, and so this kind of makes sense, and I think hopefully you can see how this all matches up. What else can you specify in Swagger? Well, quite a common idiom is to return a map from strings to something else, and as we were retrofitting Swagger specs to our services, like we kept hitting this case, and in Swagger 2 and beyond, you can actually define these very easily, and so here are a couple of examples. So a string to string map, you just say the type is object, and additional properties are type string, and you can actually change the type so you can have a string to a foo map if you've defined a foo object, and that just works. So what are these data types and formats? Well, your types, they range over integers, numbers, strings, and booleans, and then the format basically allows you to put a little bit more structure in the type. So you've got int32 and int64 for your integers, and then you've got things like dates and date times for your strings. So Swagger will actually check, if you specify date time as a format, it'll check that you really do have a date time there. But you can also have custom formats, and this is where it gets really powerful. So if you specify a format that Swagger doesn't know about, well, Swagger will kind of ignore it, but maybe in your framework you've registered a validator for your email addresses or IPv6 addresses, and you can check that the strings are of that format. And we've actually done that quite a few times at Yelp just to provide additional validation. So you've got a Swagger spec for a service. Where are you going to put it? So at Yelp, we check them into the service code base. Why do we do this? Because it basically minimizes distance between the spec and the code. You've got some other choices. You could actually put specs in a single central repo if you want it. We haven't done that, but it could work. So modifying Swagger specs. So there really is no magic here. Currently, Swagger will not prevent you from doing something really bad. You need to make sure that all your changes are backwards compatible. If you like living safely, then just add new endpoints. Or if you like living dangerously, then go ahead, change some existing endpoints, or remove some endpoints. Bad things could happen. There'd be dragons. I wouldn't recommend that path. So a brief interlude. What's the best thing about UDP jokes? Anyone? I don't care if you get them. What's the best thing about TCP jokes? I get to keep telling them until you get them. No, I'm not actually going to keep doing this. Okay, so you've got a Swagger spec. What can you do with it? There are several different things you can do with it. You can review the spec. Or you can aggregate all the specs across your organisation into a single location so everyone can browse them. Or you can generate a client library. Or you can perform some server-side validation. Or you can do some testing. And I'm going to talk about each of those now. So API reviews. So here's an example of review board. We use this at Yelp, and here's a Swagger spec that we've just opened up a comment on during review. So you really can provide a lot of feedback at this stage. And quite often we do reviews for the spec first and then we do the code reviews. And so this just looks like regular code review, which everyone should hopefully be used to. So browsing specifications. So at Yelp, we use a slightly modified form of the open source Swagger UI tooling. And on the right-hand side, you can see a bunch of different services, kind of blurred out just because I don't want to show you all the services that we use internally at Yelp. And so you can select any one of those services and then you get the API for the service in the main section there. And this actually integrates with our service discovery system so that when you actually deploy a new service with a Swagger spec, it just automatically turns up here. So developers don't actually need to do any additional work if they've got a Swagger spec. So in the Swagger UI, you can browse all of these endpoints and you can see the schema. It's great. But you can also perform real queries because this ties into dev versions of our services. So you can go ahead and put ID1 for photo IDs and then you can hit that try it out button. And it will actually go ahead, perform that request and give you a response. So this is incredibly powerful. You've got all of these services and you can interactively play with them from a single UI. And PMs love this. If devs have been working on a service, they can go ahead and actually try the service. They don't need to understand CURL or JQ or know about the kind of complicated URL structure. This just works. So brief aside, how does Swagger UI work? Well, suppose you have a host which actually hosts the JSON, the JavaScript, the HTML, and your browser downloads Swagger UI from that host. Well, same origin policy basically says in your browser that you can only communicate with a host which you downloaded that JavaScript from. But there's a problem because we actually want to go and hit the Swagger endpoints of all our services, but we're not allowed to do that. So how do we get round this? You can use cross-origin resource sharing where your services actually return a header which specify who can access them. And we thought about doing this at Yelp, but it would be quite invasive if you have to modify all your services to return these additional headers. And so we got around that by using trustee Nginx. So Nginx just sits there and it proxies all the requests both to get the Swagger UI and also to get the Swagger content from the services. And so it just sits there on a network and your browser just thinks it's always communicating with Nginx. So this is just something to be aware of with the Swagger UI. So we have a Swagger spec for a service. Let's generate a client library. So here's an example of doing that. You can run this code yourself if you want. You curl this jar, download it, and then you just run it with this generate parameter and you specify what language. I'm doing Python here. So this is a URL, the dash i option, where you can find the Swagger spec for the service and then you say output into the client lib directory. And you run that and it will just generate to you a client library, just like that. And then suppose we have this client library now, how do we use it? Well, this is Python, but it would work with Java as well or any other supported language. So you import the API client, you import Pet's API, which is defined in the Swagger spec, then you construct your client, and then you basically pass it into your Pet API object and you just say get Pet by ID of 42 and it goes off and fetches the response for that and validates it. Again, this is really powerful. We didn't have to write any of the boilerplate behind this. We don't have to write any validation code. This just works. But for Python, you can actually go a little further. So Python supports dynamic generation of classes at runtime. So at Yelp, we wrote this bravado dynamic client lib thing for Python. And you don't actually have to do that kind of two phase, fetch the spec, generate Python code, then load the Python code and run it. You can all just do it in one step. So this becomes really powerful because you can interactively discover and browse your endpoints. And you see here that in just, I guess, three lines, you can actually fetch the spec and then you can just invoke the spec just like that and you get back the same response we saw before, basically. But now it's actually, it's an object. So presumably there's a little bit of typing here as well. So something else is validation in your service. This is a project for the Python pyramid web app framework. Again, there's similar things for Java, whatever language you're using. Pyramid swagger, so we wrote this at Yelp. It supports several different swagger versions. It does validation both coming into your service and also going out of your service. And it also has a few facilities for interacting with Swagger UI. So let's just walk through an example. What do you do here? Well, you add your route as you normally do in these sorts of frameworks. And what Pyramid Swagger does is it actually checks in the Swagger spec that your URL is really there. And then when you're writing your handler code, so at the bottom of the slide, you then have access to this Swagger data dictionary on your request and that contains all of your validated data that's come in. Pyramid Swagger supports those custom formats that I was talking about. So suppose, for some reason, you wanted your pet IDs to be base64 encoded on the wire. Now, Swagger doesn't natively support this, but what you can do is you can register this validator and then whenever Pyramid Swagger sees a base64 thing on the wire, it can automatically decode it and then it can encode going back out. And here we see also there's validation, which is just try doing a base64 decoding. So let's see an example of Pyramid Swagger catching an error. So here we're hitting that endpoint that we saw before, but now we're using the string 42 instead of an integer. And Pyramid Swagger, before this even gets into your service, it realizes that this string is not an integer and it throws an error for you. So it actually says 42 is not a type integer and it says what it should be. So that's very powerful. We didn't have to write any of this code. Pyramid Swagger just did it for us. Or maybe in your pet store code, you've got this getPet function. You get a pet ID off the wire and then you're going to return an object. This isn't what you'd actually do. It's just a minimal example. But maybe for the ID we return, we make a mistake and we put a string there instead of an integer. It's a very easy mistake to make and you wouldn't necessarily immediately catch this error if you were doing something like testing or code review. You need to re-exercise the right things here. But what does Pyramid Swagger do? Well, when you try and return it, it says this ID really should be an integer. There's been an error, so it catches it on the way out. So if you just made a few example requests to your service during testing, you should pretty quickly check that you're doing the right thing because if you're returning things of the wrong type, then you're going to get an error like this. So let's talk about testing. So here's the situation if you're not using Swagger. So assuming you are using a client library, you've got two interfaces. You've got your client to clientlib interface, which is presumably within a single address space on a single machine. And then you've got your client library to service interface, which is presumably going to cross the wire. And in each of these interfaces, there could be inconsistencies. And we've seen some examples of inconsistencies before. What does the situation with Swagger look like? Well, you've got your client to clientlib interface before. But you can imagine your Swagger spec living on the wire almost, checking things as they pass to and from the service. It doesn't actually live there, but it's a useful mental model. So our generated clientlib is automatically consistent with the Swagger spec, almost by construction. Providing there are no bugs in clientlib generation, and also providing you don't go changing the Swagger spec, but we'll just kind of not think about that for now. And then you've got your Swagger spec to service interface. So let's look at the client to clientlib interface. So this is a fairly standard testing problem. You're using a third party library. You want to check that you are conforming to its interface. And so what do you do? Well, you can mock out some return types perhaps. And your type checker can probably help here if you're living in a typed language. So what would be really nice for Swagger would be if it actually supported kind of running in testing mode so you could say to your clientlib, hey, I don't actually want you to make requests on the wire, but when a request comes in for this endpoint, please return this data. So it doesn't currently support that, but hopefully that will happen sometime soon. And then we have the Swagger spec to service interface. So as part of your testing, given you've got a Swagger spec, you could use an external validator just to check your requests and your responses are consistent with the spec. Or if you're using something like Pyramid Swagger, well, that's already kind of built in, so just fire some requests at your service and it'll tell you if you're violating the spec. What else is there? So there's Swagger Hub, which is a pretty new service online by SmartBear. And here you can actually enter your Swagger specs and straight away see the documentation for them. And on the right hand side, you can also generate client and server implementations and just download them. So you don't really have to download any of that tooling, it'll do that stuff for you. And you can also share your specs between other collaborators. So this is pretty exciting because it's kind of like GitHub for Swagger. And I can see this becoming a very easy way of collaborating on APIs. So Swagger isn't the only way of specifying your HTTP JSON interfaces. There are a few other languages for doing this. One is API Blueprint by Apiri. And the nice thing about this is it's kind of designed to be more human readable and behind the scenes actually gets compiled into something a little bit similar to Swagger. So here's just an example of a pet store-like interface. I haven't used this, but it is an option out there. And another option is IO docs by Mashery. And this looks very similar, I think, to Swagger. And you can see it's just specifying an endpoint, a get endpoint. So, conclusions. So Swagger provides a really easy way to define JSON HTTP interfaces, both for new services and also retrofitting onto existing services. Once you have an interface, you get a whole lot of tooling for free. The cost of entry is that you write your Swagger specs for your services, but once you're in, there's a lot of value. So you can automatically generate client libraries for many different languages. And you can automatically perform validation of requests and responses. So the unicorn is happy now. No more tears. We fixed the unicorn world. Are there any questions? Right, so the comment is that the examples that I gave you were with Swagger specs separate from the code. Can you do Swagger specs in line with code? And I think you've seen an example of it done like that in PHP where it's actually with the code. Yes, you can do that. At Yelp, we found it quite valuable to actually have the specs separate from the code because that way you can review specs separate from any implementation. But it kind of depends on what you want. And I can also imagine cases where it's a lot easier to inline the spec with your code. Because I know that for some of the Java frameworks you can put the Java annotations on your methods saying what types it takes, what types it returns, and it will automatically generate a Swagger spec from those. So yes, that is an option. So the question is how do you use Swagger with JSON API where it envelopes the JSON response? So I'm not very familiar with JSON API, but presumably if Swagger can actually express the JSON API types, then that would work. I imagine JSON API uses pretty standard objects and types like strings and booleans and integers. And so I imagine you could code that up as a Swagger spec. But again, I don't know all the details there. So the question is, is there a way of specifying a base template that you can inherit in your Swagger specs? So there is support for inheritance, but I'm not quite sure how that would work with JSON API. So yeah, I'm not sure the answer. Any other questions? So the question is how does the workflow work? Does a PM define the spec and then hand it off to a programmer? So yeah, PMs are involved with details of the coding, but we normally have the programmers actually take the lead on defining the specs, and the PM can work with the programmers. So it's not like waterfall, PM does this, hands it over to the programmer, then the programmer does the implementation. It's much more collaborative. So the question is, are there any other tools that we use for handing off specs from PMs to implementers? I know that we still do have some written spec documents for some of our older APIs, and so those still exist. And I think there might be a little bit of automated tooling around those. So yeah, spec docs are still used by a few PMs. I think if I can remember correctly, the teams who are still using spec docs do not use Swagger. So they really are just going from the spec doc to the implementation. Any other questions? So the question is, did we encounter any disadvantages to using Swagger? So before Swagger 2.0, there were some idioms that we could not encode for existing services, so those map types, I don't think we could actually do those, which prevented some adoption. Other issues, let me think. The validation can be a little slow. It's not really slow, but it's also, it can be noticeable, especially if you're validating large data structures. And so I think one or two teams have noticed these problems, especially in something like Python, which doesn't tend to be quite as fast as Java. And I guess the other problem, if you can call it a problem, is that Swagger forces you to write these things up front. And sometimes that can be a bit of a challenge, because it keeps you honest. I would say that's a positive, but it's an additional step in the road. So the question is, are there any problems that we've had? Right. Yes, so are there any problems with retrofitting in Swagger specs to horribly designed, maybe legacy applications? So there was the map problem that I spoke about before. That problem is now fixed. I think if you do some kind of odd patterns, like if you could write an endpoint if it gets an even number, it returns a string, if it gets an odd number, it returns an integer, that might be quite hard to type in Swagger. You might say you shouldn't be doing that, maybe, but it is that it is. I believe so. Those are the cases that I have come across during retrofitting of Swagger specs, but there may be other edge cases, but I haven't seen them. So paging for results. So you would pass in a kind of an offset parameter. So that, if you want to do paging, then you can actually just pass, as part of your request, there's probably a query parameter that says, here's the offset, and here's the number of results for that offset. So it's not going to automatically make sure you're doing your paging correctly, but it will check that you're passing integers for your offsets and your sort of window sizes. That's correct. Out of the box it can do most things that I have seen. Yes. Any backlight? Right, so the question is, am I familiar with Apuri, and how does it compare with Swagger? I only have given it a brief glance, actually, so I can't really give you a full comparison. Certainly what I like about Swagger is that it seems to have a very large, vibrant open-source community, and the tooling seems to be very good, and so that was why I went with Swagger, but I can't really give you a comparison. Okay. So the question is, have I had an experience using Swagger with sort of client-side stuff? The answer is no. So right now we're using Swagger for all the majority of our back-end services, so I can't really talk to how it would work with Angular or something like that. But presumably, you know, it's still a pretty similar game where you're just typing things and checking and validating, so I wouldn't imagine it's hugely different. Okay. Do we integrate HMAC? Sort of authentication type things. Do we integrate HMAC? I am going to punt on security questions. Okay. Anyone else? Okay, let's call it a day. Thank you very much. Yes. Hopefully. How are we doing? I was a bit short. I wanted it to go on a bit longer, but... Yes. That's so funny. So what does that give you? So you get a three. Testing, testing, testing, testing. See your rupture there. The release party, can you get that out in the back? I got it out in the back. Because I'm a size that doesn't sell out immediately. Yes. Nice. That's why I brought some of the rest of the camp shirts. Just like inside reference. Okay, we're going. All right, it's three o'clock, so I'm going to start it. This is Rust system programming for web developers. Before we start out, I do have a confession to make. I am a Rustation. So if you write Python, you're a Pythonista. If you write Ruby, you're a Rubyist. If you write Rust, you're a Rustation. Rustafarian? Not Rustafarian. Rustation. This little guy on my shirt is the official Rust mascot. A little crab, a Rustation. That's what I do at night. If you write PHP, does anyone know what you are if you write PHP? That would be misguided. Actually, I kid, I write PHP during the day. I've been doing it for ten years. But I love Rust. I feel in love with this language. I work at www.northschirmrack.com. I hope to look during the day writing PHP and managing, and I write Rust sort of on the side. To Rust itself, let's actually talk about what system programming is. There's no definition outfit that everyone's going to agree on about what system programming is. And those of us in the Rust community seem to have coalesced around this definition of system programming is when it matters. So I'll give you an example of when it doesn't matter. I'll pick on Ruby this time. Ruby is slow, but it doesn't matter because the database is a bottleneck. So when you're writing Ruby code, we're not calling that systems programming, right? Because it doesn't actually matter if Ruby's a little slower than where you're writing it in C because your bottleneck's the database, right? There are a couple of characteristics that you're going to sort of see if you're a system programming. You're not going to have a garbage collector. You're going to have to manage memory yourself. You're going to find great control over your memory management. So we see that a lot in C or C++. And you're going to have zero cost abstraction. So that basically means that you're writing code that's sort of against the metal or against the hardware, right? You could be flipping registers on and off if you want to. You can represent a data structure in the exact size of memory that you specified. You can make entire programs sort of allocate memory on boot and then allocate nothing afterwards running just on the stack and then free up at the end. So you have a lot of control. So that is systems programming. And let's talk about why I think Rust is a great systems programming language and why I think web developers should take a look at it. Let's start with some history. Rust is a language that's sort of been developed by Mozilla. It was developed with the primary use case in mind. Mozilla runs, as you all know, the Firefox web browser. And since the web is such a popular place nowadays, there's lots of events like Pone to Own, where a bunch of security researchers or hackers will get together and they will present their exploits. And they do this with Firefox and Chrome and every single browser out there. And Mozilla sort of knows this pattern, where all the exploits were basically coming centered on this sort of one vulnerability, which is a data race. And that sort of happened when they would sort of have all these sort of threads running. They write C++, all those threaded programming going on to make Firefox fast, even though Chrome is faster. But they got all these threads going on and there would be sort of some mistake that someone would make with the way the threads are working and sort of sharing memory that would cause a data race. And hackers would take advantage of that and make the web browser do things like a remote command exploit or they would make it do something like going to a web page and fetching a resource that it shouldn't. And this was driving Mozilla crazy. They wanted to fix this. So Rust's primary motivation is to sort of fix this problem that Firefox is having about sort of not having some sort of safety checks around this because threader programming, even for the most experienced developer, is super hard. So what they did is they developed, they're working on developing now a web browser engine called Serbo, which is sort of a replacement to Gecko. So if you don't know, Firefox uses the Gecko rendering engine to actually render out the HTML and the CSS on your web page. So they're not really building a new browser, but sort of building a new engine. We actually already see elements of Rust and sort of kind of being feathered into the existing Firefox browser. So before we get too far too, we're talking to a bunch of web developers hopefully. Has anyone here actually written a Rust program before? Also a fair amount of Rust stations in the room. That's great. Welcome. But if you're a web developer, you might be asking yourself this question, is Rust good for web development? And the Rust community sort of has this penchant for making these websites called like www.areweblankyet.com and www.areweblankyet.com You can actually go to this next, talk about writing a web browser, or perhaps a web service in Rust. And you can do this, but I think it's too early to tell if Rust is going to be good for web development. Rust is primarily a systems programming language, so we're talking about that. And really the problem with Rust in the web development context is that the web is made up of strings. HTTP is a bunch of strings coming in. Your database connection is usually a select with a bunch of strings coming back out to your Ruby or PHP or Python code. And really converting all those strings to types normally isn't worth it, right? Web requests, we hope, are under like 200 milliseconds. We're trying for expressiveness in our web code. We're using GC to kind of save ourselves our memory management a lot of the times. So it's just really not worth it to sort of do that. Also, we don't really need fine, great controller of memory because, again, we're only 200 milliseconds is our span. If there's some memory leak or some memory overhead, the request is going to be finished up quickly anyways, and we'll start over again. Also, Rust is a compiled language, so there's a compilation step. So we're not really going to focus on Rust for web development. I'm going to sort of slant this over to why web developers to sort of look at actually getting involved in a systems programming language. Yep, so Rust plus web developer I contend can be a systems programmer, right? So let's talk about this. I think I did it actually. Skip a slide. So Rust sort of provides these three constraints, right? Memory safety without garbage collection, concurrency without data races, and abstraction with the overhead, is sort of what makes up system programming. And we're going to sort of go through these and how they can be applied to web developers. So I sort of have this basic theory, right? Anytime a new programming language comes out, a bunch of people will get on the internet and they'll say something like, oh, this language is going to replace all the use cases for this other language, right? We saw that with Rust too. Rust was sort of coming out, it was eight years in the making. It's at 1.6 now. People were sort of discussing a lot, oh, do you think Rust is going to replace CRC++? I don't think that's really actually the case. If you're writing CRC++ right now, you're probably pretty comfortable in a language that you've been writing for years and years and years. And you might switch, but a lot of people, they have no reason to switch. They're working a day job using CRC++ and they sort of invest all this time of energy in getting familiar with that and they can sort of work through that. I actually think that Rust won't be a CRC++ killer and I think instead web developers are sort of the largest group out there right now of people in the software developing community and lots of new people coming in. I actually think Rust is sort of well positioned given some of its features to actually be attractive to web developers and actually grow the overall pie. So there's really not a focus on actually cannibalising that existing blue graph blue pie chart area of CRC++ developers, but I'm actually arguing like, I think Rust actually can put a nice slice of new systems programmers into the community, which I think is sorely needed. I think there's some historical context to this, right? I just didn't make this up out of thin air. We've sort of seen this. So has anyone sort of was programming when Rails came out? And were the same thing happening where there was a bunch of debate going on in the web development community talking about like would Rails, would Ruby on Rails sort of unseat PHP as the dominant web language? Would it sort of take over and you see these blog posts flying back and forth? Rails is faster and PHP is faster and sort of it's never ending, right? And there are benchmarks, benchmark slide. But what no one sort of understood at the time is that it's infancy, right? Like EC2 starting to sort of get more popular. There's other competitors out there. We're virtualizing so many things. You have a whole generation of ops people that aren't even, they don't even know how to work in the cloud, right? They're used to like procuring some hardware, like setting it up, you know, with some custom bash script, maybe they're using CF Engine. And then like that thing's just going to run and we're going to pretend like it never like shuts off. And if God forbid it dies, then you know, there'll be some Horos going on. And what happened with Ruby with the Ruby community, right? Like they're wanting to, Rails is new, people are coming in, they want to actually get this stuff in the cloud and there's no tools for that. And then the Ruby community finds out that Ruby's a great DSL and they start writing the tools themselves. And all of a sudden you get things like Cap Astronom for deployments. You get Puppet and Chef 4 as a much better replacement to CF Engine for configuring servers. You have multiple sort of ways of running multiple Ruby versions in development and even God forbid production if you wanted to. You have awesome developer tools like Vagrant which is a Ruby script around virtual box that makes life much easier for us and things like Sensey which is a Ruby replacement for Nagios. So all these great, so Ruby didn't create the DevOps movement but certainly what they were doing, they found that Ruby wasn't replacing PHP. It just actually created this entire generation of DevOps people that have built a bunch of tools that basically everyone in web development community uses or has been influenced by. So that sort of happened. We kind of see it actually going on now, right? You take a UI developer, slap some note on them, all of a sudden you've got a full stack developer. But why is this, right? So I was a PHP developer when Node came out and I remember looking at Node going, why? Why would anyone want to do this? And a bunch of other people who were writing Python or Ruby said the same thing. What's going on here? And Node is a new technology, right? Like nothing that Node sort of brings to the tables brand new. And then he would have rendered JavaScript on the server for years using Rhino. We've been able to do async.io using LibEV, I think, was that what it was before LibEV was created and used in Node. So there's no new technology here. It's been around at least a decade. Why all of a sudden is this such a powerful movement? Like what's going on here? And you just sort of think about like web's eating the desktop, right? Now if you're a UI developer, your requirements being given to you are like, I don't want to laugh and everything dynamic. I don't want to reload the web page. Just make everything like single page apps, age apps load everything. So these UI developers are writing HTML and CSS and JavaScript are going to the Ruby person and they're saying, hey, I need you to give me a lot of this event driven sort of interaction. I need you to do a lot of this stuff for me. And they're going like, what are you talking about? I know rest. That's all I do. And they're like, no, I want web sockets. I want all kinds of nice interactions. And there's this friction. So with Node, all of a sudden you took a UI developer who was fighting with their back in engineer and you're like, hey, here's all the tools. You can just do the whole job yourself. You already know JavaScript. Here you go. So now all of a sudden you have a full stack developer and it's happening right now. And for a long time the server side people just didn't get it. You saw blog posts like Node.js is a cancer. People putting up blog posts like, hey, Fibonacci runs really slow in Node. And as we all know now it's more of a mature ecosystem. Like it's really powerful. It's really good. It complements a lot of what people are doing out there. And it didn't. So no one lost their jobs doing PHP or Ruby or Python development. But what you have is a whole generation of people who are only working in the browser, now working on the server side as well and providing a lot of influence in the community. So I think the same thing is going to happen with Rust, right? You're going to actually have web developers and you're going to sort of convert them into systems programmers. And that's not to say you're going to stop being a web developer. You might actually just do both. You might have to sort of enhance your tool set. And why do I think this is needed? So as we all know, like 10 years ago we were building monolithic applications, right? One repository, a web server, maybe your runtime behind that. And if you want to scale that out all you kept doing was horizontally scaling, right? Spinning up more VMs, procuring more hardware. Unless you like the .1% of the industry that's basically how you scale. Throw more databases behind and read clusters, right? Now even the average engineering shop has microservices and all these different parts of their application. And if you start going through some of the concrete examples of each of these, you're going to find none of them are actually written in the languages web developers know. Yet we're highly reliant on them. So if we're talking about application caching like Memcache or Redis, both written in C, HTTP caching layers, barnish and squit, job queues. I think rescue might be like the one example here if you're in the Ruby community where you have a Ruby sort of implementation of a good job queue that you can use. If you're in Python or PHP, you're using something like Gearman or Rabbit. Relational and most equal databases, these are all written in CRC++. Search databases, the next other exception here, those are written in Java, though any op person hates this and high workloads in the GC pauses on you at your most critical time, you're getting really mad and so you fight with it non-stop and every op person wishes that these are written in CRC++. Load balancers and VM in containers, like we're all like VM in dock rising everything under the sun right now, right? Again, all written in a non web language. So we're highly dependent on this entire ecosystem of software that web developers have basically very little access to. It has this massive knowledge gap, right? It makes it worse that all this stuff is open source, so there's really no one getting paid to do this stuff, which means you have very, very few people actually working on this open source software that actually have the know-how because it's CRC++ and this is bad for a few because it's one like we've seen burnout in the community. Also we've seen that a lot of these maintainers don't have time for things like important ancillary tasks like documentation, or putting out proper releases and things like that. It also hinders the state of the art. When you don't have people sort of like fresh people getting into a certain area sort of a couple winners emerge and then things sort of stagnate, right? We saw this with the web browser for a while where the web developers weren't actually involved in the web standards. You get a bunch of CRC++ people that were developing browsers sort of figuring out how APIs and the web worked and when they would come out, all the web developers would say like what are you doing? That does not match your use case at all. Only recently do people in the web community have they started joining TC39 and even getting things like ES6 and ES7. Like that wasn't an accident folks. Like it really took web developers getting involved. I think likewise, we're missing core use cases in the web community because we're not getting involved to solve parts of our web stack for ourselves because we know best. There's a clear example as I write PHP during the day all the time. PDO is the database adapter abstraction in PHP. These PHP data objects. The person who wrote that is a C programmer. He doesn't write PHP and the way he wrote it in the constructor of PDO when you do new PDO with people normally doing the bootstrap, that actually makes the network connection to the database. Like no web developer wants to make a connection to the database unless they're explicitly saying it. You have to work around this as a PHP developer. Everyone has to do it. It's crazy, right? It's because Wes wrote a great library but he missed a core use case because he wasn't actually involved in the PHP community. I'll go ahead. I'm sorry. I think for the video, same name, I'm overselling it. I'm laying out the background of why web developers are getting involved in Rust now. We're going to go through certainly a lot of code examples but I thought the history was important to present. There are some barriers to entry for web developers to get involved in systems programming. Why haven't people and this is why I want to set this up. Rust is fundamentally different here. Web developers aren't getting involved in C and C++ because it's unapproachable in my definition. There's a problem with memory management. When you can compile your C program and it compiles fine and then you put it into production and all of a sudden it seg faults on you at the most critical time you're not going back. That's a super painful experience. If you're a web developer trying to get something done and try to get this critical task out that's not making you look good. C or C++ provide that foot gun for you. You can't rely on GC if you're doing systems level stuff as we discussed. I think a lot of web developers don't want to write C or C++ because it's not an expressive language. When we write Ruby we like how expressive it is. When we write Python we like not having to do semicolons and we shouldn't have to give all that up if you want to write systems programming. A lot of times we don't need types and we're going to custom not having types. Rust can solve a bunch of that stuff. These are the three tenets of Rust and we're going to go through and talk about how they allow you to do memory safety give you fine grain control which is your cake that's like fall. We're going to show I'm not getting too much into concurrency because it's hard to show in examples but we will get into a lot of abstraction without overhead. I do want to make sure everyone understands I think 2 is super important here too because as we know right now we're taking a single piece of hardware like a server and we're breaking that down into virtual machines and Docker containers. I think at some point pretty soon here in the future we're going to see a lot more use cases and we're going to see multiple machines trying to make them work in concert together and right now trying to do that if you have architecture that's horizontally scaling you sort of know that that actually is a lot of time and effort to make sure those things are coordinating correctly. Having concurrency without data races is super important I think for the future where a lot of this system stuff is going. Let's get into some of code examples. Rust right out the gate has basically this concept of ownership of have memory safety without garbage collection. Everyone are going to give us some examples of how ownership works. I'm going to start walking you through Rust syntax with the examples if any of the syntax doesn't make sense let me know. In this example we're defining a function take that has a single variable v of type a vector of signed 32 bit integers and then below that function we're going to actually define a variable v that is a vector of 1, 2 and 3. We'll pass that variable v into our take function and then we're going to print out below that the first element of v. In any other language that you've probably been exposed to this is perfectly fine perfectly sane and perfectly normal. It'll compile, it'll work, it'll run. In Rust this is an error and all it comes down to ownership right we are actually passed our v into the function take and once we did that we have given up control on that scope using that value v again I'll show you the code again. So line 3 once we pass v into take we are no longer allowed below to use v or access a d anymore and this fundamentally there's references and other stuff that Rust introduces here but this fundamentally is how Rust gives you memory safety without the garbage collection piece here. It basically ensures at compile time that you're not doing anything that would basically cause problems and this is really nice because it solves a whole class of problems that C or C++ don't solve for you. You don't have segmentation faults anymore because you're not basically allowed to sort of use after free essentially right or even use before you even allocated it. The compiler is sort of using this rule called like the barb checker this barb checker is going through and basically making sure at compile time you're not doing anything that would basically blow up your program at run time and you also don't have memory leaks anymore because Rust is sort of handling the freeing after it's allocating it's also handling when it's supposed to free so you don't have to worry about this whole notion of I need to have functions that are dedicated to allocating memory and then making sure I'm only freeing that memory and then nothing else is sort of accessing it right and if you've written C++ before you know this is sort of just a chore to do and Rust does it for you for free. The problem is when you start out with Rust you're going to start fighting the program because as you saw that was a perfectly valid program and we all know that that's how we write code and Rust doesn't let you do that and so when people start writing Rust they normally start fighting the barb checker and they're like that's perfectly normal I do that in this other language why won't you let me do that and really what's going on here is the sort of the barb checker isn't doing anything actually that you shouldn't be doing if you're actually a grizzled veteran C++ developer you basically have internalized these conventions of the barb checker and what you just sort of know it's tribal knowledge to you you go and read some of the best C++ development books will outline these steps makes you do all these things and what Rust did is codify that into a compiler so what Rust is when you're fighting the barb checker you just take heart and that you're kind of making an investment and that the compiler is literally teaching you painfully at times because it doesn't care how frustrated you are and it doesn't actually go on vacation but it's teaching you to become a systems programmer and you could actually take those same rules and apply it to other systems languages and be just as successful one thing I actually like about this too is you could contribute back with confidence right how many people have sort of wanted to put a patch into a large program open source and it kind of felt a little uncomfortable about actually pushing that pull request button right because even like the test pass like you're like I don't know like it might blow up at one time I might have missed something right and you know it's kind of scary especially if you're new to that community or you're a new developer even if you're seasoned you look like a knucklehead if you just sort of miss some obvious thing right and the great thing about the barb checker even though you're going to fight it at first you're going to get to the point where like you know that it's going to work and if it does blow up it's not going to be for some obvious thing it's going to be for something kind of crazy and so like I myself have contributed to RustCore and I'm not like that good of a programmer but once the thing compiled I just felt like I can push the button here I can actually send this patch over the wire because like the compiler told me I'm a good systems programmer for this particular piece account Cool so great so now we've this is why Rust is so compelling in terms of writing save code but you know I did talk about like we don't want to give up some of those aspects of scripting languages that we like right so let's get into some of these things that Rust does that we don't look like we're actually writing so much sort of CRC++ code so Rust spends a lot of time on ergonomics and we do think writing the code should be as enjoyable as writing it in a system scripted language and we want the compiler we use the compiler to figure out all this borrow checking stuff right all this ownership stuff and we also use the compiler to figure out all this sort of boilerplate stuff that you might do in other languages that have strong types and we sort of let the compiler in for it you know I can example the canonical example here is like in Java right you have to define the types on both sides of the assignment it's like why you find it on one side just figure it out on the other side and what we end up doing right is instead we produce IDEs that basically fill in all the boilerplate for us right and instead of using IDEs Rust let's just stick it in the compiler so here's an example simple example we're going to find some variable foo that's going to be a typed signed 32 bit integer and we're going to sign it to the variable 5 and as humans we can all reason about very quickly like that's a number I don't need to specify the type so neither does the compiler and this looks like a lot of languages that are like scripting languages right but foo equals 5 no type necessary that's great for like type like assigning like variables of types we also can go one step further the compiler can even figure out in certain cases how to even default the return type so here we get a little more complex this is how Rust defines a closure so we're going to find some variable f and it's going to define a lambda function that takes in a single variable x of type signed 32 bit integer and it's going to return a signed 32 bit integer and it's going to return x and this is an example of the identity function if anyone was curious and this is how you write a closure in Rust or a lambda function in Rust it's pretty terse we can even get better that is how you can write that same function in a lot like a scripting language so you can go through a lot of your Rust code and not actually worry about specifying the type and that's nice because sorry go ahead so I'm going to say that by actually writing it so not only is it nice not have to write the type in certain cases this is actually more expressive even to your code because you can use the same function to actually allow multiple types to come through and the compiler will know that so it's even actually better like this is encouraged late as programmers we're sort of taught laziness is a good thing and here it's showing up in the positive and we're going to a little more complicated example these apostrophe things are called lifetimes and so what we're doing here is we're defining a function that's very similar to our identity function called silly and I'm introducing this concept I'm introducing a lot of concepts here but 5% means reference and that lowercase str is a string type one of the string types in Rust so we're passing in some variable s of type string and as a reference and passing that reference back out and we see like these apostrophe is all of Rust's functions require you to sort of define these things called lifetimes and that's how it actually figures out how long things like references should live for it's all really complicated you have to be like you know some programming language theorist to actually figure out a lot of times when certain lifetimes can be used but the great thing about Rust again we're using the compiler like 90% of your code does not need the lifetimes at all the compiler infers it this is actually really nice because we get all the power of lifetimes without having to worry about it so if you're a new developer and you're trying to do something in Rust you don't want to have to start thinking about lifetimes to try to just get something done right and so the default here is lifetimes are there under the hood but they're just sort of like inferred for you and when you want to actually turn on like the advanced switch like you can switch that advanced switch on and start telling Rust no no I know exactly what I'm doing here in my lifetimes and be really explicit about what you want to do so you get all the control with sort of none of the penalty right at this point we sort of kept it pretty introductory and I've been introducing a lot of things that are probably pretty basic we're going to sort of step up a notch here to a little more theoretical stuff that's going on anyone has any questions please stop me and ask so optional correctness yes go ahead so the question is is the Barrow Checker reference counter no it's not the Barrow Checker is a program that by default there's no sort of reference counting going on we're literally at the middle level where like C is it's just a couple phases in the Rust compiler this is the first phase that it goes through and just sort of make sure the semantics of your code aren't violating sort of any of these constraints but so it's not a runtime thing it's done at compilation time lifetimes think about lifetimes as basically a it's less of a scope it's almost like typifying sort of how long reference to a variable should live lifetimes are one of those things I introduced it it's going to come up if you're getting into Rust you don't need to actually worry about them hardly at all but you kind of want to know what they are in case something in case you run into something when you're using a reference I would say most people have written Rust in the go months without ever needing to put one lifetime in place so they're there be aware of them don't be scared of them optional correctness one of the things I love about Rust is it sort of takes this nice balance of theory to actually real world problems and as we know the real world is not always sort of on our terms and so I like a language like Haskell it's great but you may have to jump through a lot of hoops sometimes to make your programs work and I think Rust strikes this really nice balance I think we can sort of great example this balance is in Rust you can do really return types right so we've all seen this type of code we're going to let R equal the value of some function return value of some function if R is an error we're going to return early return false and then we'll go down our happy path code right we've all write that when we write like our normal code and we write that in Rust too but Rust also has this monadic type right we're going full burrito now this monadic type option that can wrap a type T so any type T it can wrap this and if you're not familiar with option it's like the either type in Haskell option can basically say look if there's something here of type T or there's nothing here right so it kind of maintains these two states and if you're going to go to programming language theory sort of side of things this is a monadic type and monadic types are not allowed to return early Rust is like whatever you can return early so what we have here is an example let variable foo equals some string bar and we're going to use the find function on foo to see to find the character B right and this match statement in Rust it's kind of like a switch statement right but it makes sure like you sort of cover all the different use cases so as I said before option is either there's something there or there's nothing there so if there's something there, some character C there then we'll just return C back to our variable so C you know is the is where that position is and we're good and that's what this code is going to do right for example where our string is no and B is not there and we're going to go into this other branch of code, this other form of our match statement where it's not and we're going to call this macro panic which basically means like fail error, bail out of your code stop finding the code it basically return early type so you know when people sort of if you're a programming language theorist you see option like I love monads but Rust is like look you can return early too if you get all the power of monads it's sort of all the simplicity of sort of returning early which is just easier right if you don't have to like sort of contorting your code to say like okay I get only one return type at the very bottom of my function scope you're trying to figure out how you can break your functions even smaller functions to deal with sort of the different if else you know statements that need to happen it really makes things much more approachable and this pattern right here is so common in Rust that they actually made it even easier to do to have this method called unwrap so this is the same thing as before that basically unwrap will say look it if you found what you were finding I'm just going to return it to you and if you didn't I'm just going to blow up the code and I'd be like man why would Rust put that in there I thought you said this was supposed to be like memory safety and like really good systems programming stuff well it's true but if you're starting out in Rust or even starting out trying to solve a problem you're not worrying about correctness right you're trying to hack your way from start to finish and being compiled is something generally working to see where you're at right we don't normally explore new spaces by doing TDD first right TDD is after you've already sort of mastered your Ruby or your PHP or your Python TDD is when you're sort of just trying to you know you're on rails if you're starting out Rust or sort of exploring some unknown problem where you're trying to figure out like some weird protocol and you're twiddling bits you don't want to be dealing with TDD right you don't want to be dealing with really correctness either you want to turn that off and so you can use unwrap it's fine if it blows up it's just it's not production ready yet that's okay and you can go back later then and actually say I'm going to now actually handle like errors properly you know log them you're going to bubble them up you can do all these different things so it's really nice again a pragmatic approach and really I think it goes down to Rust sort of says like the programmer knows best like we're going to sort of put all these defaults in place and we're going to sort of set this really nice language up but the programmer knows best and we'll sort of allow them to sort of override things when they want to so much to the point that Rust also has this notion of unsafe where I sort of told you before you had to satisfy the Barbara Checker if you wrap coded unsafe Barbara Checker says I trust you so here's an example where we're actually creating the sparkle heart emoji out of a vector of bytes and we're saying like look at Rust trust us we're going to call it UTFA characters if they're not then that's on us and what this is sort of nice is like again it allows us if we're trying to really take a theory and apply it to the real world sometimes the real world is not in our terms and we need to do something crazy right in this case it's sort of we're sort of demarcating like I need you to trust me right here Rust and also to all the other developers out there like hey I need you to trust me here Rust doesn't have my back here I'm saying I got this it costs some about zero cost abstractions what I sometimes call mullet driven development you're going to get all the sort of the expressiveness we're actually all of it a lot of the expressiveness of Ruby so you get sort of the Ruby in the front when you're writing your code but you don't take any of the performance costs you really get like C in the back right this is great so let's go through some examples here right here we are defining some variable n and then taking a range of numbers one to ten we're first mapping them we're adding those numbers by two then calling filter on them so any number greater than five we're dropping and then we're adding those all up with our fold function here with an accumulator this looks very similar to something right like in Python or Ruby right and this is like if you're like you know cutting edge Ruby is at this point like you're like no for loops ever I map everything it's great you can sort of do the same thing in Rust but if we wrote this in Ruby right you basically loop over every single one of those numbers one to ten once from app and then we're going to loop over the numbers again for filter and then we're going to loop over the numbers again for fold and that's going to be basically three n times now if you write Ruby again it doesn't matter because that's not what you're trying you want expressiveness you're not trying to optimise your code for performance you want expressiveness, maintainability, readability and we take that hit we don't care but if you're writing a systems program you do care so it actually this is Dallard Rust code what Rust is going to do under the hood is take map filter and fold instead of doing those three iterations it's actually going to create like a single loop and actually do all that work in a single path through of ten iterations it'll actually inline map filter and fold and sort of do that in sort of one shot and you know so you don't need to worry about it like if you want to write your code expressiveness and make it really clear what's going on you don't have to go back later when you want to optimise it and sort of make it almost unreadable has anyone ever looked at the OpenSSL code we just before? like you wonder why there are so many bugs in that thing they're trying to make this thing fast and when they do that it becomes almost unintelligible what's going on here we can make it really clear what's going on yet still get all the performance out of it could you add more methods like that as a user of the language so the nice thing about Rust is Rust is written in Rust and these are just iterators so you can write your own iterator methods and sort of do this yourself and people do this all the time for the question the question was why does filter take a reference instead of the actual type the reason why filter takes a reference is it doesn't actually need to own the variable it just basically needs to sort of use the variable to check something real quick unlike map map might actually mutate the variable filter is just sort of peering into it so you don't actually take the variable you just take a reference to the variable and that's common in the generic filter method so these are all like a mutable thing so you can make map take a reference if you really wanted to other questions and then we can even go one step further where Rust really does have first class functions like a JavaScript type language I wrote some code I created a simple reple re-evaluate print loop for Rust and this is what my main function looked like again systems language re-evaluate and print are all functions and again the compiler using LLVM compiles is down and we're taking no performance hit and we're getting sort of all the monadic stuff going on in here with the admins we have all the expressiveness like it's very easy to see what I'm doing here let's go one step further with generics if I define a function here first and if you're familiar with like Java or anything like that we're defining a generic type A and a generic type E and creating a tuple out of those values and we're basically just saying it's going to return that type A and the body of our function we're just saying for that tuple just return the first one so that's what the function is doing Java Java is actually figuring out at runtime like how to actually figure out all the typing stuff and you can run into some problems there where it can actually not figure it out so for Rust we can do the same thing like you can with any other generics we pass in a jersey, you can pass in strings and you know even when we do this and something like Ruby or PHP or Python it's doing this at runtime we just never see it, it's a hidden cost actually what Rust does is it has this concept of monomorphisation long word don't want to remember it but essentially what this does is the compiler creates a specialized function at compile time for each type so if we're passing in if we're passing in an integer the compiler goes oh I see you're passing in integers in your program I'm going to go and create a specialized function for integers if you're passing in a string it's like I can see you're passing in strings as well so I'm going to create a specialized function for strings I can look basically like this like you can do this yourself too you can write specialized functions Rust does this for you for free so you can write really generic code we saw with our closure example earlier and under the hood it's creating these specialized functions for you that you don't have to manage it's just like I got this for you so again all the expressiveness of sort of a higher order language with none of the sort of the performance cost the question was could you have a string and a numeric I am defining two discrete generic types so you could I didn't show that example but yes you could kind of do any sort of generic type that you want and then let's move into another topic here so systems programming we might think if you want to write become a systems program you've got to write the entire thing in sort of that language it all has to be in C or all Rust or all C++ I mean doesn't sort of the definition doesn't actually require that what we can actually do still is write the bulk of our program in something like Ruby and then once we're done writing that program we can actually go back is there any slow bottlenecks that are not IO bound vcpu bottlenecks and if there are we can actually drop Rust in to actually solve that problem I actually did this there's a library out there called selecta which was written by Gary Bernhardt which is a really nice sort of little command line utility that allows you to quickly search your file system for things it's really nicely written and has a full TTY and all this support but there was this one hotspot in the code where there's this really efficient search algorithm but because it was Ruby it just could never run fast and it was always like the biggest bottleneck on the code I actually just took that library rewrote the search algorithm in Rust and put it back in to the overall program in Ruby so Ruby could just call my Rust code and we can do this because Rust treats C as the common language across the entire ecosystem which is basically true and Rust interrupts the C really well I'm not showing an example here because it's a fair amount of code to get it going but there are a lot of examples of this on the web on my blog post I have one for PHP again I've written one for Ruby and it's approachable there's some boilerplate work to tie into those C interfaces that are sort of defined but Rust makes it pretty easy because you have all that fine-grained control Rust isn't like a GC language if you try to do this in something like Go which goes a great language but Go has a GC and if you try to match Go's GC after Ruby's GC what do you think is going to happen like not good things because Rust doesn't have a GC just like C or C++ you can easily tie into this even Node so what recently came out to was an example from someone at the I remember his name now at Mozilla actually put a whole blog about how you could actually tie Rust into the Node.js runtime which is written in C++ a little bit harder than C so I went over a lot of concepts it's really hard to go through some of the code examples here but there's a lot of ways you can sort of start digging into Rust there's actually a Rust book out yet but the documentation on the website is great Steve Clabnick has done a great job he's actually put up Mozilla and does that full time but I actually think even better we learned Rust is to come to the Rust at Los Angeles meetup, I actually run that meetup and we do things like last month we took there's a new program out there called Stockfighter, anyone's heard of Stockfighter before and we it's basically a game where you write an automated stock trading program and it's pretty real, it's not 100% real it's pretty real and believe it or not I've written a Rust SDK for it so we're writing a Rust SDK and really I think the hardest thing about with any of this stuff is getting started right Rust lowers that barrier to entry to get started a lot but there's always sort of something that doesn't make sense for some of the people those questions get answered really quickly and you actually feel like you're making a lot of progress there's the website on there and again reach out to me if you have any questions that link at the bottom is a link to all my blog posts on Rusts and there's actually a number of them on Strings, so if you do start writing Rust and Strings are frustrating you go ahead and check those out it should make things a lot more clear but feel free to hit me up on Twitter or anything if you have questions as well and with that I'll open the floor generate a Rust executable to whatever you need or are you not able with EPR8 or some other small yeah you can the question was can you generate a Rust program for something like Arduino Rust recently in 1.6 which just came out actually has stable support for not including even the standard library of Rust only just like the bare of the core components which allows you to compile it for things like the Raspberry Pi or Arduino and basically the footprint is very light you get just the bare bones and then it is, I realize that you still can do for the Arduino the Rust compiler has compiler targets and you have fine grid control over the different compiler flags that you can send and make and you can sort of compile it to anything you can compile C2 and the question was how small can the Rust binary be Rust Hello World is a little bit larger than you might think because again it's doing a lot of that sort of specialization under the hood again if you need to optimize something you can do it unsafe I don't remember off the top of my head but I know it's a fairly large Hello World they have not tried to optimize for the Hello World example I know some people have been interested in that but it's kind of a red herring and once you get into something that's beyond just a simple program it sort of gets amortized out and you're basically sort of on par with CRC++ don't quote me on that you could certainly ask on the Rust users forum and they would be able to give you some direct examples I think there's been some talk about that so I'm not into that space as much so I can answer definitively but certainly it's of a lot of interest to the Rust community on that Other questions? Joel and then there is a Rust libraries or user libraries in Rust are called crates there is a Rust crate that's starting to replace parts of OpenSSL I think you know Rust is 36 weeks stable they went through eight years of basically prototyping to solve that main core use case that I've served up so we've been 36 weeks in like a 1.0 release and we're in 1.6 now and I think really the community is sort of looking at things like a 10-year framework reference so I think we'll see more adoption of those things as they come but sort of things are kind of going slow as we're sort of figuring out things like even things like higher kind of types I think as we sort of figure out more and more things within the community you can see more adoption and see things like like an OpenSSL port to Rust or other things like that and we'll start with a library called MIA which is a replacement for LibUV so it uses things like Apple or KQ if you're on BSD but it's a complete Rust replacement for LibUV I think Dropbox just put out and that's what they use to sort of create their hard drive mesh and to figure out how to load balance all those different files across your system so it's coming and some people are really interested in doing that because it's nice to write in your question So the question was can we talk about concurrency control on Rust yeah it's I would have to dedicate like ten slides I think sort of to walk through just how that sort of works under the hood there are some examples up but essentially Rust is using the Barochecker and you can sort of reason about how you're allowing threaded computing to share memory via the Barochecker so it does that all at compile time so it's really using that those same semantics to sort of enforce that you're not going to step on a piece of memory so once one thread takes ownership of a piece of data the other thread just can't you don't need semaphores or new texts at that point right and then you can actually use a lot of other semantics at like when things get freed up or out of scope to pass back ownership upstream to a different thread so like there's a really good way you can like do that with channels for example it's really easy so the question was is that better than Go? so Go is focused more on their like Go routines right and they do have channels and stuff I believe the last time I checked the Go routines actually leak a little bit of memory in order to do a lot of their safety guarantees so again Go is a great language I think it has a lot of use cases I think to like replace a lot of Java stuff for like web servicing it's like kind of lighter weight than Java I think that's where it really shines and again you don't really care about if it's a little bit of leakage there right but um there are cover routines in Rust you can do the same thing but they just sort of have different strategies again Go has the garbage collector other questions here then we'll go back to you what is the next Rust yes other questions what is the next Rust meetup and where do we meet the next the meetups are usually in the middle of the month so it's whatever that Wednesday in the middle of the month is and we meet and we meet in the Norsham Rack slash Hope Look Productions Studios in Culver City and if you go the website has like a huge thing of direction so you can get there yeah check out the website it has all the information hope to see you there question yes so competition is good I actually don't think it's really gonna impede it too much you know we've even seen like in C++ a lot of stuff that we've been doing in Rust the C++ community has been trying to do to like take on as well there's actually now some sort of certain pointer types that sort of have some kind of ownership and properties that Rust has been sort of messing around right if there are already like things like own pointers and C++ I think Swift is gonna sort of make it more again make things more accessible we're gonna get a lot of people who are writing iOS apps starting to write actual like server side stuff like in the USX ecosystem so I think actually it might only speed up adoption of something like Rust they have a lot of similarities but they also have a number of differences too that the Swift community is like we're not we're not trying to compete at that level that sort of some of that we're competing some of the areas where Rust is trying to compete another question so the question is does if the question is if the difficulty of functional language semantics would be a blocker for the adoption of Rust so Rust actually isn't expressive language not a functional language so it actually can do a lot of functional things that you find in other languages but every single thing that you do in Rust is an expression I mean by that is you can actually assign a variable to the result of an if statement which is actually after a while quite natural so you can write your code in Rust in the most basic imperative form that possible so it's great like almost in that classic C-stop right which is a bunch of functions and that you just sort of cobble all together and make a program out of it so again a lot of the stuff that we talked about is opt in you don't have to take advantage of like map and filter and fold if you don't want to you can actually just write just normal functions and it'll work just the same so I actually don't think that's too big of a deal I think it's really more of an up so that it can do those things cleaning most people sort of want sort of a higher order feel to the languages that are working and now the question is does Rust have exception handling so Rust does not have exceptions it does have a macro called try which at first glance might look like like deal with exceptions that's just bubbling up things like options or result types but the language designers made an explicit decision not to sort of add exceptions into the language panic and though things look like exceptions as well like they'll sort of quickly abort the program but yeah I don't think there's any plans for exceptions in the language as well that's sort of a highly debated topic but that was the decision any other questions all right well thank you so much for listening to talk I hope to see you at the meet up so you guys are here to bust the language but it's not a good way of talking to speak to you so you're trying to cater it fast and you don't have to say it like that word better than just all of us because it's fine just want to go back and I'm looking for a hand to bring this back and I'm actually a blogger but I should be able to do that because I'm not a blogger so I'm trying to do that because I'm a blogger but I'm not a blogger so I'm trying to enjoy the content of the language but it's not a good way to put it. I'm not a blogger but I'm trying to do it because I'm a blogger because I'm not a blogger but I'm trying to make it easier for people who are talking to me and I'm not Yn oedd eich cyfnod, ond y llunaf, yr wych are fawr yn fawr o amser yn ei wneud o'r amser a'i cymhysigol. Mae'r ddiddordeb yn cael ei bod yn gweithio'r ddiddordeb yn cael ei ddiddordeb. Mae'r ddiddordeb yn cael ei ddiddordeb yn cael ei ddiddordeb. Mae'r ddiddordeb yn cael ei ddiddordeb yn cael ei ddiddordeb yn cael ei ddiddordeb yn cael ei ddiddordeb yn cael ei ddiddordeb yn cael ei ddiddordeb yn cael ei ddiddordeb yn cael ei ddiddordeb yn cael ei ddiddordeb yn cael ei ddiddordeb yn cael ei ddiddordeb yn cael ei ddiddordeb yn cael ei ddiddordeb yn cael ei ddiddordeb yn cael ei ddiddordeb yn cael ei ddiddordeb yn cael ei ddiddordeb yn cael ei ddiddordeb yn cael ei ddiddordeb yn cael ei ddiddordeb yn cael ei ddiddordeb yn cael ei ddiddordeb yn cael ei ddiddordeb yn cael ei ddiddorde Rhaennau ar yr ysgol yw'r hanffl, i'w parhau sydd datblygu i'r ysgol ffwrdd aneth. Felly, er mwyn amser, ddiweddio ar y sylfaen ar gyfer mynd i'r ffordd i mi ddaeth. Diolch o amserau a substitute, wedi cael ei wneud sy'n mynd i'w gweinyddadau am amser. Fe hynny, rwyf wedi bod y dyfodol yn gweinydd y byddol yn teimlo i'r byddol. A rhaen, rwy'n yr ysgol fyddwyd wedi gweinydd. I have a bit of reputation for building talks around me making fun of things, and this talk is no exception. The difference is that every stupid thing that I'm about to list in this entire presentation is something I've done inadvertently. So if you can't laugh at yourself, who can you laugh at? So there's no shame in coming and admitting that you can't necessarily do something. So let's start at the very beginning. Who here uses git? That's an impressive showing. Who here wishes they use git? Who here is just raising their hand because they're staring at their phone and want to pretend they're paying attention? Good. Good. It's hard. Git is extremely simple. It tracks changes to files and directories over time. It lets us collaborate with each other and share those changes back and forth. It lets us work on teams that are distributed both in distance and in time. But it's extremely simple, which is why in these talks the next slide invariably looks something like this. Because, let's face it, there's an elephant in the room when it comes to git, so let's introduce it. It doesn't matter who you are. Eventually, git leaves you feeling like a moron. It gets very deep very quickly, and the only question is how long you can hold on for the ride before you've gone off the deep end. And it seems like there should be some reason behind it. Why does git get this complex? Acropo of nothing, git was written by Linus Torvalds, who is famous for three things primarily. Yes, he did indeed. He names all his projects after himself. We'll see what the third one turns into. But in any case, at this point in time, Linus Torvalds is famous for three things. Right in git, which is why you all are presumably sitting in this room. Creating Linux, which is why everyone attends this conference and why everyone in this room more or less has a job. And for tearing people to pieces on mailing lists. So git, despite its complexity, git does let you do some incredibly powerful things. Powerful, of course, being a euphemism for stupid. You can commit large binaries to git. You can commit atrocities in git. It tends to operate relatively, sorry, build repository first, shouldn't we? What an amazing concept the demo got. Get in it. Local operations on git are really fast. So when it hangs like that for a little bit, that's your first indication that something might not be quite right. And not as much of a genius as I thought I was. Okay, so now we see that in addition to committing an ISO. And we have built out the .git sub-directory that tracks all of the changes that go on in here. And if we take a quick look at that directory, it's now 69 megabytes. Which is a totally reasonable thing to stuff into a Docker container, but that's neither here nor there. So, okay, I've done something stupid. I've committed a large binary to git. That's okay. The git status tells me the file is on track. It's no longer being watched by the git repository. Huzzah! But notice we still have a 69 megabyte .git repository. Changes in the past, things that have been committed to the repository, stay there forever. It's awesome. Yeah. So the way we're not often committing large binaries to git, but what we do, and we want to fix it, in order to want to wind this, you've really got two options. You've got a relatively complex command called git filter branch, which you can wind up going through and removing all references to it. And then you wind up having everyone else realise, hey, there's a bit, there's a mismatch between what you think you have and what the server used to have. You're rewriting history when you do that, which is bad. Or you can use a Java tool called BFG, which will go through and tear through things. It takes a little bit of time and it's annoying. Now, here in the real world, if you want to go ahead and store large binaries in version control, look into two different projects. They're designed to solve this problem and they do it a lot more effectively. Gitanax was written by Joey Hess, a former Debian developer, and Git Hub wrote Git LFS, which they are pushing harder and harder all the time. Now, most of us aren't going to be spending time committing binaries into git, at least after the first time we do it, and get yelled at by management. But we're often committing things that we shouldn't be, like secrets. Replace large binary with an API key for Amazon or an SSL certificate or a password, and you begin to see the problem. Now, at Future Advisor, we are committing secrets into git. We're just encrypting them with GPG first, and then using GPG render for Solstack to turn those back into plain text. That's a sensible and reasonable way to get to work with it. But remember, you can go back in time just because it's not in the current revision doesn't mean it's not there somewhere in the past. There are stories about people exploiting systems by doing deep dive searches on Git Hub and back on Google Code when that was a thing. Let's move on to a terrible idea. I don't even know how to describe this. Let's go ahead and show you. That's how we learned to type. Remember how I said git can make anyone's head hurt? So, git in it. It creates a .git repository. If we take a look inside, we have different things that point to the latest commit. It keeps track of different branches. There's a local repository specific git config. The usual things here, but these are all files that wind up controlling what the git repository does. Well, these are files. I care what happens to them. I should probably track them in something. The sad thing is that this works and you can go deeper. If you're ever doing a fine and things like this hierarchy turn up, something very wrong has happened. Okay, on a topic here. Let's take a look now. If you've ever done this, you start typing a command like git status, for example. Then you have to turn away and look somewhere else to actually copy the top repository, whatever it is you're looking. Then you come back, git status. There it is. Git's telling you you're a moron. There are ways to fix this. Now, what's in that file? Very simple. Just git dollar ampersand. Sorry, dollar ad slot. Good Lord. You can tell I'm losing it. It winds up working really well. You can now git status. But wait, there's more. And it works. Something very similar to this will make docker docker docker a reality. It is turtles all the way down. So git does have its own alias support. You can build git aliases. But that doesn't always tend to go quite far enough. Because for starters, you have to preface all of them with git. So if we don't want to do that, we can back out into the shell and wrap shell aliases around lengthy and expensive git commands. For example, why in the world would I want to do this? This is the right place. There we are. You accepted this talk. Is that it opens new frontiers in being a jerk to the people you work with. I really feel like this should be on the GitHub's marketing page. They can have the slogan, I'll license it by a Creative Commons. So we've all heard about rebasing master. What that effectively does is it goes back and changes history. So everyone working on a repository suddenly doesn't have the history and the thing they're working on that they thought they did. It changes history so everyone else gets screwed. If you see a group of ravens, they're called a murder. A group of whales is called a pod, not a docker, thank you. A group of developers is called a merge conflict. Not a developer, big surprise there. I work in operations and I don't have time for this. So I use this. It's a lot simpler. Bob's note, we're just the department of note. It's easier. Okay, so we've shown that git can make us feel stupid. Why don't we let git help us feel a little bit smarter? I really wish there was a better way of a Mac to transition more elegantly than that. Okay, ever type of something like that and then git tells you, hey, Jack, didn't you mean this other thing? Well, there's a way to fix that. That command does and I'll break that down for you in a second. It's actually kind of helpful except for the part where it can screw you if you're not careful. But that's computers, right? This is more or less a type what I mean, not what I say thing. Now what this is, we're setting a git config option. Note the dash dash global. A small detour, there are three places that git config will alter things. Globally, which winds up doing this in your users.git config in your home directory. The system level, which is an Etsy git config, which is a really great way to confuse anyone else on that system. Or by default in the repository itself. So that's kind of awesome. What the help.autocorrect does is it says, go ahead and if there's only one right answer that you think I'm trying to type, go ahead and use it. Now what the eight does is that is the number of tenths of a second it will wait for you to hit control C before saying screw it, I'm going anyway. We sometimes call that the penalty of having to wait because you type out something. Set that high enough and you can make it really painful. It's kind of awesome. But it doesn't work on everything because this winds up only working when there's one right answer and it's not as smart as you would like it to be. For example, how many times have you tried this? So you add a remote repo, things are great. You want to push your review, you want to push to where it's going. And it tells you that no, if you want to push it to the same name branch upstream, you have to explicitly tell me and it's frustrating. Fuck. And then it works. And it works again. This is a program that's been making the rounds. It's on GitHub most notably more than any other technical thing that they have done. They have managed to get the first Google result when you Google the phrase the fuck. That's amazing. Okay. So you are aside, I do have three legitimately useful tools that I want to talk to you about today and they do tend to play rather nicely with each other. So who here views management as perpetually looking for something shiny like a moderately intelligent squirrel? I see a few hands including some people who work for me, that's good, good, good. As a manager of my son, I have to say you're not far from wrong. So when I first started at my current employer about six months ago, they didn't really have anything resembling a monitoring system and we fixed that. But while it was coming up, we had a big monitoring board next to my cubicle that was dark. So what we wound up, what I wound up putting up there instead was something very similar to this. It's a program called Gorse. And what that does is it lives in a Git repository, you are in a Git repository, it's an open source software program, it lives in just about every different type of, in any type of package management system you've got for every operating system. And what it does is it provides a visualization around every bit of history that happened in that repository. So we would see fun things like our CEO way back in the day going ahead and making some changes. And then our founding engineer would go in and immediately revert all of them. And it became a great, it was a great way of looking at where we had come from, where we had, where we've been, where we're going. And once we replaced it with a monitoring board, people didn't come by and look anymore because they would just like to come and tell stories and marvel at the history of what we have done. So that was kind of nice. Don't ever doubt the power of being able to show something shiny, even if it's not as informative as you would hope. G-O-U-R-C-E, it'll be on the board at the end, it'll link to all the tools I'm talking about. So does anyone have more than one Git repository that they care about? Let's do work at Facebook two years ago, your brand's probably on top. They were famous for having a single monolithic repo. And it sucks because when you have 15 or 20 different repositories that you care about and you get on a plane and you realize you're missing the latest changes to that one repository that you really need in order to get something done, like put your slides together for a conference talk. So Mark Atwood posted a decent fix here. Specifically, you just punch a find command into a for loop to find Git repositories, iterate over it, you fetch the latest changes in every repository's parent and I would think half of you just fell asleep. So this is really awesome except for the part where it's shitty. Because I'm going to do this once, but I'm not going to go ahead and run this every time. So I tweeted back at him. So I took his stupid loop, but instead of just running a Git fetch, I ran a different command. In this case, MR register. And then I don't have to remember it anymore. And he tweeted back, this is awesome, thanks. And the point here is that we're always learning. Mark is the director of open source at HP, maybe HP Enterprise now. I haven't been keeping up with the various changes. And this is a guy who's forgotten more about software than I'll ever know. We're still always learning about this one weird trick that no one has pointed out to us before. We're also usually not the first person to encounter a particular problem. So when you start realizing this is annoying, there's probably a better way. Ask people. So let's talk about what MR is and does. It used to be called MR in the tradition of open source projects of having a name that's virtually impossible at Google. And then they said, this is actually a really handy tool. Maybe we should change the name, which makes it easier to find. But now I never know which is the proper name and I invariably use the wrong thing. They've managed to take one of the harder problems in computer science of naming things and made it worse. So step one is you can run MR register inside of each repository you care about. You can do it by hand. You can do it using a for loop like the one that was on the board a few minutes ago. And it doesn't really matter because you only have to do this part of it once. It goes ahead and it builds out a file called dot MR config. A typical Unix fashion, one of your dot files. Then you go ahead and run MR update and watch as it updates your repositories. Specifically, I give it MR list. It spits out a lot of things that didn't show up because of the weird resolution on this. Then I tell it to MR up, which if the conference Wi-Fi behaves, there we go. It starts in series updating each and every one of the repositories that I care about. If I take a look at my MR config itself, note that there's an include there at the very top. Note the include at the very top where I'm including my entire dot MR by MR config directory. Which means I can drop individual files into there. So if I have, let's say, something for the front-end team and there's a list of repositories they need, I can build an MR file and just pass it out to them. They can wind up using this. Also, if you take a look at specific environments, for example, you take a look at Puppet or Salt, you see a bunch of different repositories that in turn wind up being given specific instructions of where the parent lives. The list is not working well with TeamMux at the moment, but it spits out 30 different repositories, which is really the relevant part of it. You can also break this down into groupings by going into subdirectories. If I go into my workspace directory and do an MR list again and type it properly, in turn it breaks it down to 12. I've never seen it do this. There's something strange with the projector anyway. So at this point, that winds up letting you sub-target among different groups. There is a slight movement toward getting actual better grouping logic handle-ness, but it hasn't gone anywhere yet. Patch is welcome. And it's not just working with Git. It works with virtually every version control system that you can think of. The notable exception that's not on this list is Perforce, but Patches are welcome, which in the open source community is how we say, go away, Git, you bother me. I mentioned how we can limit operations by using a sane directory structure. I tend to like hierarchies for things like that. We're on the front end directory where I just have those work repositories and a back end directory, and they both live in a work directory, so I can just go drill down a lot more effectively. Now, I wouldn't know that this tool existed if someone hadn't told me about it, and that's really what part of the community aspect of open source comes down to. Now, a Stodgy German friend told me that it existed, and I thought, this is ridiculous. Why would I ever use... Oh my God, that's kind of amazing. Now, I bag on them a lot for this, but Docker absolutely nails this. You can't get people to stop talking about Docker, and that's kind of fantastic. Anyway, my crazy German friend told me about this in conjunction with a tool he himself had written called VCSH. This is effectively a version control system built on top of Git for your home directories. Clear as a global .files repository where they keep their things like .zsa, .bashrc, .vmrc. Right, it's a common thing, and it usually looks a lot like this. There's countless numbers of them on GitHub, and it's fine, but there are aspects of this model that are kind of crappy. I don't want the Git repo that has my personal AWS credentials in it. I keep those in Git, I still do terrible things. On my work machine, and vice versa, I don't want my work credentials from a Git repository living on my personal machine because that's a great way to get fired and sued. So what I do instead is use something called VCSH. Okay, the listing function in this works at least. What it does is it lists out a different series of repositories that are controlled in this. Let's, for example, take my Git config as a great example. I run this, it then winds up instantiating a shell session underneath where I am now, and if I take a look at where I am, I'm still in my home directory. I haven't changed anything about that, but it does take advantage of how Git works internally to wind up with a Git root that lives somewhere else. And if I do a Git status, my .git config has been modified, but there have been all kinds of file changes in my personal home directory that are not being picked up. The reason that happens is if I can't not get ignored, I have a wild card in it. What this means is that in each repository I do need to explicitly force at any file that I want to track over time. That's a little bit of a hurdle, but it winds up paying for itself later in time. The idea is then I have my Git config living in a completely separate repository from my SSH config, and neither one of them know anything about each other. I can make my changes. I'm ready to go. Then I hit Ctrl-D to log back out, and I'm right back where I started in my home directory. I run Git status now, and it doesn't see it as being in a Git repository, which is handy. It will also avoid the submodule problem among other things. So what this does under the hood is you have your home directory, and right under that you have a .config directory. This is stolen from some of the Debian people. They tend to like this particular approach to things. I'm Luke Warmon of myself, but it works. The idea being that we then wind up spreading things out, so you have a list of things that are available that VCSH can manage, and you can pull these with MR, which is typically how you would handle this. The ones you want on this machine usually have a sim link in your config.d directory that wind up pointing back to the original source of the repository. So you can effectively turn repositories on and off with this. Therefore, because I've never pulled things except for what's linked in config.d, while I know where those other repositories live, I never have their data locally. Then, in turn, when I wind up checking out the repository, it does under the VCSH sub directory in repo.d, which then is where that repository actually lives. But checks out its files into my home directory. I have one more trick that I do want to show you that has been extremely handy as I've gone through Git. A lot of people have something that does this already, but surprisingly, never done, so I'm going to point this out. Now, you've been seeing it throughout the entire presentation, but I haven't called attention to it. You'll notice that next to my prompt and the ridiculous, dockerized hostname that's a hex character that winds up being populated, it tells me that I'm on my Git master branch and there's a red one with a dot next to it. What that tells me is that there's a file that's been modified and it's been added to Git staging area, but it's not yet been committed. So I go ahead and I make that commit. Now my prompt has changed. You'll see that it still has a green check mark which tells me that things are decent, but I still have a commit that needs to be pushed up. Notice the one with the up arrow next to it. It tells me that, Git status tells me that I'm ahead of origin master by one commit, use Git push to publish those commits. What having that in my prompt does is twofold. One, I have to type Git status a whole lot less, and two, it gives me a constant sum of reminder of maybe I'm not on the branch that I think I'm on. If I'm about to make a commit to a feature branch and I see the word master there, that should set off a quiet alarm bell in my mind. Now, increasingly, if I first built the original version of this talk, Git Hub and others have made it harder to forcibly push changes to master if you disallow it on a particular repository, which is great, but not everyone enables that, not everyone can enable that due to various workflows, and ultimately it tends to not be particularly a panacea, which is why I do like having that subtle visual clue. So here's a link at this point to everything that I mentioned today, and I'm going to leave that up there for people to take pictures of, jot down while I answer any questions you folks may have. I keep thinking they're hands raising, but nope, just pictures. There we go. Hey, a hand. Unless there's a camera in it. What's up? To repeat the question, I was asked what graphical interface am I used to manage Git repositories? I come from assist admin background. I use the Git command line client, and that's it. Other people like things like tower. Some people like, I think, Git K is in somewhat frequent use, but personally, I've never made the leap into using a graphical client for it. That's mostly preference at this point. From my point of view, I can always drop into the command line tool if I can solve that, then what the graphical elements do don't necessarily matter. I do understand that's a highly opinionated perspective for me to have, but unfortunately I'm the wrong person to ask. Anyone have something they use for a graphical product they love? Yeah, last year I did have something I know. Is it source tree? Yes. I think someone at the office might use that, but I wouldn't swear to them. Sorry? Magic? Fugitive. Oh, well. Vim fugitive. Yes, yes. Yes. Yeah, you can do that too. Ultimately, this is one of those things we can wind up about. You can take a survey of a room like this and wind up and spend the next 20 minutes listing things on a board. There's a lot of answers in that space. Find something that works. If that's the direction you want to go on, I'd suggest finding something that works for you. Try them out. Of course. Other questions, comments, concerns, criticism, snark. Hit me with it. Well, you know one of my favorite things to do in the open source community is disappoint people. It feels like I'm standing through a room full of my dad. It's great. Hobson's choice I was given was sub modules or sub trees. I really don't like either of those. Sub modules are annoying specifically because it's not always obvious what they're doing and how they work. Sub trees are supposedly the better way of doing things. Personally, I avoid it with things like VCSH, which if you leverage it correctly and move it away from just a home directory, you can use something like that that works a bit more effectively. But it's one of those awkward questions that everyone has when you ask them because there really isn't a good answer to that. For those who aren't familiar with what sub modules are, is you put a git repository inside of another git repository, not in the crazy way that I did inside a .git, but if I go down a level and then create more repositories, how do you wind up keeping the parent repository informed of the changes in the child? Yes, what I use for that is something called ZSH. It's a ZSH specific plugin, which I'm noticing not as many people are using down here as they are up north. So I tend to not necessarily call that one out specifically. Some people love Powerline for that. Other people use Aeropin, and there are a few more that go along with that similar naming convention. I'm sorry? Bullet train? That's one of those situations again where there's 50 different answers to that question. But again, the way I picked the one I found was I just started playing around with it one day and moved through a series of different ones that I found when I liked. If you're using ZSH, feel free to tweet at me, my Twitter handles on the next slide, and go ahead and dig out the one that I'm using and get an answer for you. Other questions concerned? You again. Good question, a little off topic, but are there valid reasons not to use ZSH? Well, again, a lot of what I, the same reason I don't use a GUI, forget is because I spend a lot of time logging into systems that are not directly in front of me, and in order for me to use things that I'm comfortable with, I should at least understand Bash. So it mostly comes down to portability. If nothing else, Bash is everywhere, because ZSH is not. Other questions, concerns, snark? Thank you all for coming. Yeah, it's hard to describe regardless. Five or six different machinations that people are not using here, because they find out five years later that they started with SBM, so we have to do a massive integration around those. But as soon as you've changed it at one, the other one is going to stay. So it's going to stay. The conference is now officially over. Felly, mae'r wneud yn fawr i'r fawr.