 Hello. Is my microphone on? Yes. Thank you everybody for coming. My name is Andrew Gerand and I work at Google in Sydney, Australia, and I work on the Go project. So what is Go? Who here, first before I start, who here has heard of Go? Okay, who here has read the documentation or looked at the website? What about actually written some Go code yourself? Alright, cool. So for those of you who don't know, Go is a general purpose programming language. We build it as a system language when we first announced it, but it's turned out to be useful for a wide variety of things, not just systems programming. I've got a list of its killer features here, the things that sort of make it special in combination. So the first of all, it has a strong emphasis on simplicity in its design. It's very easy to learn. All of the language features are very simple in and of themselves, and they're also orthogonal. The way that the language features interact is very easy to predict and very simple to understand. It's a memory-managed and syntactically lightweight language like a dynamic language like Python or JavaScript. It has that kind of easy, loose feeling where you don't have to worry about memory delegation and you don't have to do so much worrying about typing things. It compiles to native machine code in Intel x86 32-bit and 64-bit code generation. In practice, its speed is comparable to C. We have two compilers. We have GC, which is our custom Go compiler, and we have GCC Go, which is a Go front-end for GCC. The code generation on GCC Go is slightly faster than GC, but GC is the more mature compiler that we devote more attention to, but they're both feature-complete. One of Go's big feature points is that it has strong concurrency primitives built into the language, and it's a CSP-style concurrency model like other languages like Limbo, if you've heard of them. It is a statically typed language. I mentioned that it's kind of like a dynamic language, but it's not a dynamic language. It's actually very strongly typed, but it has a nice system of interfaces that make a kind of compile-time duck typing possible. It's a lot more malleable than your traditional statically typed languages. We have a very consistent and small standard library that is really nice to use, and all Go code is self-documenting, and we have a documentation extraction program called Godoc, and therefore the project is very well documented. We place a huge emphasis on complete and correct documentation in all of our projects code. Finally, it's a free and open-source project, even though the core development of the project is sponsored by Google. The core team are all employed by Google, but we release everything in the open. It's under a BSC license. All of our development happens in the public on public mailing lists, and we've had over 150 different contributors contribute code to the project directly who aren't from Google, and of those people, there's probably about 10 to 20 who contribute on a regular basis. We actually have a Windows port that is almost, it's like 95% of the way there, and that's being contributed wholly by non-core contributors. We've got a good community growing, and we're one year old as of November, so it's pretty exciting. What am I going to talk about in this talk? I'm going to go through the complete development of a web application, a simple one. There's a lot to cover in this talk, and so it's going to move quite quickly. It's going to contain a lot of code examples. If you're not familiar with Go syntax, there will be things in there that aren't immediately clear. If you're the type of person who sort of obsesses over small details, I encourage you to kind of force yourself to look past it and focus on the actual sort of semantic meaning of what I'm explaining rather than the precise syntax. We'll just get started. If you're going to tweet about Go, then use the Go lang hashtag so that I can find it and see what you've got to say. The program that we're going to write is pretty much the simplest of all possible web applications, which is a URL shortener. It does two things. As a web service, it takes long URLs and then returns shortened versions of those URLs. Then when it receives a request to the shortened version of the URL, it will return an HTTP redirect to the long URL. The best place to start when writing any program is with the data structures. Our program Go to will store maps short URLs to long URLs. The natural data structure that we would reach for is a hash map. Go provides a built-in map type that allows you to map values of any type to any other type. To initialize a map, we use the built-in make function. This is a line of valid Go code. This is the actual type, a map of integers to strings. Our variable M is now a map of that type. To put the string one in the box identified by integer int one, it's just the familiar syntax that you would see in many different languages. To pull it out is the exact opposite. You would now equal the string one. The first bit of unusual Go syntax for encountering here is this multiple value return. This expression M2, we can actually extract two values from it. The first value would be the value stored at two. The second value is a boolean value that indicates whether that key is actually present in the map or not. In this instance, there isn't a two in that map. V would be an empty string and present would be false. If I put a comma present after the U, then present would have been equal to true. Now that we've decided that we want to use a map, let's specify the actual type that we're going to use for our program. We'll call it URLStore. This is a Go type definition. Basically, just giving a name URLStore to this concrete type, a map of strings to strings. To instantiate this hash map, I just use the make function and then M points to that region of memory at which that hash map exists. To store a mapping of A to Google.com, I just assign Google.com to the key A and to pull it out is the similar reverse. This is all we need as far as data structures, but it has a shortcoming. The map type isn't thread safe. If we want to write to the map for multiple threads, we need some way of moderating access to that type. Because we're going to be serving a web application, by definition, it has many concurrent requests coming out at once. To protect the map type from being modified while it is being read, we must add a lock to the data structure. To do this, we'll change the type definition instead of just being a map of strings to strings like that, we're going to turn it into a struct. A struct is just like in C, a concatenation of values of other types. Any Go type can fit inside a struct. In this case, we have two fields, URLs, which is our map of strings to strings. The second field is called Mu. It's of type sync.mutex. Sync is part of the standard library. RWmutex is a reader-writer mutex. Multiple clients can take the read lock simultaneously, but then when somebody tries to take the write lock, only one person is permitted to take it at once, or one function. That's at the exclusion of all readers. In order to access the map and be mediated by the lock, we need to add setter and getter methods. The get method will take the read lock by calling MuR lock and then returns the URL of the string after unlocking the read lock. This is our function definition. The function name is get. This is our function receiver, which is on a pointer to URL store. And then its first argument is a key, which is a string, and it returns a string. So first we call S MuR lock, because S is our URL store. And then we extract the URL from the map, and then we unlock the read lock, and then we return that URL. And our set method does the reverse, but with a slight added complication. The set method will take the write lock and update the map. But if the key is already present in the map, we want to return false and not do anything to the map, because we don't want multiple clients to be able to clobber each other in writing the same URL to the same key in the map. And this will come up later and make our algorithm simpler. But in this case, if to walk through this particular piece of code, set takes a key and a URL, both strings, and then returns a Boolean value. So first we take the write lock, and then we extract, and then we find out whether key is present in the URL map. And if it is present, we release the lock and we return false and bail out. But if it isn't in the map already, we can store the URL under key, and then we can unlock the write lock and return true. And this is a little verbose, and it could be cleaner. And to demonstrate how it can be cleaner, I'm going to introduce this concept called defer. In Go, you can use a defer statement to push a function call onto a stack. And the stack is local to the enclosing function in which you make the defer call. And you can defer many, many function calls. But then when the surrounding function exits, all of those function calls are executed in last in first out order. So for this particular function foo, I'm deferring a call to print line from the string formatting package. And I'm deferring a print of world. But then I actually just make a normal function call to print hello. And so foo, when it executes, will defer that function call, print hello, return. And then this function call to print world will occur, printing hello world. This is pretty much the simplest way to introduce defer. I don't have much time to go into it in detail. But the go blog, if you search for go lang blog, there's a really thorough article written about this that interests you. But we can use defer now to simplify our setter and getter methods. So our get method from before, if we look back a couple of slides, it used a temporary variable URL. Instead now with defer, we can just take the read lock, defer the release of the lock, and then return the value from the map directly without having to use an intermediary variable. And similarly, in set, we can take the right lock and then immediately defer the release of the lock so that whatever happens when this function returns, that lock will be unlocked. And this means that we don't have to include unlock twice in this function before each of the return statements. And so this is the most common use of defer. But that's our set and get functions. Now to complete the design of our URL store type, we need to initialize it whenever we want to use it. Now I mentioned earlier that we need to make map types before we can use them. And so because URL store contains a map type, I need an initializer function that will initialize that map and return a new URL store. Now this would be a place where in typical languages, you might expect to use a constructor to construct it. But Go doesn't have this concept of constructors. Instead, we just use functions of the form new foo, where foo is the name of the type that you want to construct. And this seems like an omission, but it actually ends up being quite useful when you're writing more composable code. But I'm not going to go into that today. So our function new URL store returns a pointer to a URL store value. And in the body of this function, all we do is we have a URL store literal. And then inside, I specify that URL is the result of making a map of strings to strings. So this function will return a URL store that is ready to use. The read write mutex doesn't require any initialization. It's zero value is a mutex that is ready to use and that is unlocked. So when we're using the URL store, if I want to initialize s, I say scorn equals new URL store. And now s is the type pointer to URL store. And then I can call s.set to set a key. And if that returns true, then we've successfully added it to the map. And then to get a URL by key, I call s.get. And you notice that this if statement looks a bit strange. In Go, if statements can have an initializer in them in the same way that a for loop can in many languages. So in this case, the variable URL is the result of our get call. And then in this if clause and any subsequent else clauses, that variable URL will be scoped to that particular set of blocks, which makes it very easy to keep things nice and clean and compact. So it's a small bit of Go idiom. So now that we have our data type, which is thread safe and ideal for storing the data associated with this program, we now need and we already have a get function, which will serve us fine. We need some way of putting a URL into the map and getting a key back in exchange. So this method put takes a URL, generates a corresponding key and then stores the URL in the map under that key and returns the key. So in our the way we'll generate the key is by the way most URL shortness work is that you take some base, like base 63 is a common one where you have upper and lower case, alphanumeric characters and zero to nine. And you just take an integer and then convert it to that base. So I have this function gen key, which does exactly that. But I'm not going to show the implementation of that here. And then I have added a new method to URL store called count, which simply returns the number of elements in that map. So Len is another built in function. So we're taking the length of the map returning the number of items inside the map in a thread safe way using the read lock. So I generate the key by generating the corresponding alphanumeric key associated with the corresponds to the count of what's in the map. And I do this inside a for loop, an infinite loop. And so what I what I'll do is generate a key attempt to set it. If the set succeeds, then I return that key. That's a success. Successful put. If it hasn't set it, it means that somebody in the meantime has already generated that key and set before I have. And so I loop back around and generate the next key. And by this time, count will have incremented because somebody has already inserted something under my feet. So we've talked about how we have gotten put, which service the back end part of our URL shortener. But now we need to look at the user interface. And the user interface is going to be served over HTTP. So we'll use goes HTTP package to provide the infrastructure for serving HTTP requests. Now this program here is a complete go program that serves the whole world over HTTP. And just to walk you through it briefly, we have package main, which is a package statement. Every go program begins with a package statement. And every go program begins in package main with function main. So I'm in package main, I import the string formatting package and the HTTP package. And then in my main function, I call HTTP handle func, which registers this function hello to handle all requests to slash, which is our web route. And so when somebody accesses the web route on this server, this function hello will be invoked. And it has this function signature that takes a response writer and a request. The request is the request data of the HTTP request, the incoming request, with information such as any query parameters, the path itself, the address of the person making the request, any headers, et cetera. And the response writer is a type that you can write to, which is what I'm doing with fprintf. And then it also has a set of utility functions to modify headers and other things. But in this case, all I want to do is write the string hello world using fprint to w. And then finally I call listen and serve to set up my web server listening on port 8080. And then this function will block forever serving HTTP requests. So how can we leverage this to actually implement go to HTTP interface? Well, our program needs two HTTP handlers. It needs a redirect handler, which will take incoming shortened URLs and provide an HTTP redirect to the longer version of that URL. And I have the add handler, which handles the submission of new URLs and returns the shortened versions. So in our program's main function, we'll use handle func to register these two new functions redirect and add under the web root and slash add. And the semantics of this is that the request to slash add will go to the add handler and then requests to anything else will get sent to redirect. And this is the behavior that we want for this program. So here's the implementation of add. The add function, it reads the URL parameter out of the HTTP request. So a form input box named URL, for example. And then it puts it into the store and then sends the corresponding URL back to the user. So inside our function, we get the request form value that's called URL and store it in the variable URL. And we call store.put, putting the URL into the store and getting a key in exchange. And then we print the new shortened URL, which, because I'm running on my machine on port 8080, you'll just get, I'm just printing that out with the key appended to the end. And we're writing that to the response writer. So that's what the user will see in exchange. But what is this store? Well, store will be a global variable that I've created. And somewhere in my source file, I have this at the top level, vast store equals new URL store. And so this will be the global instance of store that will be accessed by the add and redirect HTTP handlers. But I only showed you one part of the add, the post handler. We have to have the part of the handler that displays the form in the first place. And so if URL is empty, it means that there's been no submission or there's been an empty submission. And so in that case, I'll just F print this constant add form, which is a string constant of an HTML form, and print that and then bail out. But if URL is not empty, it means that somebody has actually submitted a URL and I can throw it into the database. I mean, and this is an overly simplistic UI. Obviously, if you're going to build a proper web app, you would put a bit more effort into it. But at this stage, this actually works quite well. And so the redirect handler is a function that takes a key out of the HTTP path and then retrieves the corresponding URL, the longer URL, from the store. And then performs an HTTP redirect to take the user to the site. If the URL isn't found, we want to show a 404 error. So in our redirect handler, we access the request URL's path and we slice off the first character of the path because it's always going to be a forward slash. And then we look up the key in the store with get and take the corresponding URL and if the URL is empty, we know that it's not in the map. So we return a not found error using this not found helper. Otherwise, we can redirect the user using the redirect helper and we redirect them to the URL that we retrieved from the map. And we return a status of found 302 found, which is the appropriate HTTP code. So we've written, like, I think it's about 90, something of lines of code. And it's a complete functional working web app. And I can demonstrate what we've got working so far. So I run that, head to localhosts8080, the conference Wi-Fi. OK. And then if I want to add, my finger's a cold. Is it cold in here? So I add FOSSTEM and then I get the resulting URL. When I visit that URL, it should take me to the FOSSTEM site. And then I can add another URL, growing.org. And we'll see that this works as well. So that's pretty nice. When this process ends, when I hit Control C, those URLs that I've entered have just disappeared into the ether. So it's not a particularly useful application at the moment, unless you have some magical computer that will never switch off in a process that will never end. So we need to modify the URL store so that it actually logs URLs to disk and then restores that data when it starts back up. So before I get into that, I'll just introduce another aspect of Go's type system, which are its interface types. And interface types define a set of methods. And then any type that implements those methods can be used where a value of that interface type is expected. So in a way, you can think of interface types as defining behaviors. And then any object that behaves in that way can be used where that interface is expected. So one of the most frequently used interfaces in GoLand is the writer package, the writer type, which is specified in the IO package. And so the type writer is an interface which contains the method write, which takes a slice of bytes or a buffer and then returns the number of bytes written and an error value. There are many, many different types that implement writers. There are compressors, network connections, HTTP connections, files, tar archives, you name it. It's in there. And in fact, we've already used an IO writer in our HTTP handlers. The W response writer argument to our HTTP handlers, it implements the write method. And the fprintf function in the string formatting package expects an IO writer. So we can use the response writer as a writer because of the shared interface. So that's interfaces. But we were talking about how to represent this URL store data on disk. Well, the way we'll do that is we'll use GoGob package. And GoGob is a serialization package which turns Go data structures into streams of bytes and vice versa. The GoG package has the function's new encoder and new decoder, which wrap IO writer and IO reader values respectively. And then they return encoders and decoder objects which allow us to call their encode and decode methods to write and read Go data structures. And so when we're writing to disk and we want to store this data on disk, we need a data type to describe it. And so we'll create this new type record, which is a struct that has two string values, the key and the URL. And then our new save method, which takes the key and URL as a string, will create a new gob encoder, which wraps some file, and then calls encode on that encoder to write our new record. This is a record literal, which contains our key and our URL. And then that encode will write that record to the underlying file. But what is s.file? s.file is our new field file, which is a file handle that I'm putting on the URL store. And then in our new URL store initializer function, we'll add this code, which has gone off the screen. To open a file with os.open, the file name is a new argument to URL store. And then if the file doesn't open correctly, then we'll bail, because if we can't open the data store, we probably don't want to continue running the program. So log.fader will actually exit the program. But if we do succeed, we'll set s.file, the struct file field, to our file handle f. And so then in save, when we refer to s.file, we're referring to this open file. But now the last part is to load the data back into the program when the program starts. And so we define a load method. And this load method returns an error value. It takes no arguments. And first what it does is it seeks to the beginning of the file. And if the seek fails, it will return an error. And then it creates a God decoder. And what we do is we set up an error value. And then we go into an infinite loop. So for error equals nil, it means just continue looping while there isn't an error. And then I set up a record value r. And then I decode into r by giving the decode function a point to r. And so decode will read bytes from the underlying file and try to marshal them into a record struct, into this record struct. And if that succeeds, if error is not nil, we can call the set method to set it into our map. And in this way, this will loop forever until it reaches the end of the file. And when it reaches the end of the file, error will equal os.eof. And if that's the case, we want to return nil. We want to return a no error because that's the expected failure case. But if there's any other failure case, we'll return that error and pass it up a stack. And so the last thing we need to do is stick a call to s.load into our initializer function. And if that fails, we'll print a benign error. Because the first time we run this program, our store won't exist. And so it's OK for this to fail on the first run. And finally, we need to add a call to save each time somebody successfully makes a put. So in our put function, before returning the key, we'll call s.save passing in the key in URL. And then if there's an error with that, we'll log it. The last thing is we need to, when we instantiate our store in the top level of our program, we need to provide some file name to where the data will be written and read from. So I can show you what we've written so far. So if I add Google.com and then visit slash one. And you notice that I actually already had one stored in there, slash zero. It's the FOS stem side. And so if I control C out of that, if we look, there's this store.gov. And if I hex-dump it, you'll see that there's two records in there. The first one is a key of zero pointing to FOS stem.org. The second is a key of one pointing to Google.com. And it will continue on in this vein. And so if I restart the program and reload this URL, it should work. But it's not. Don't worry. There's a lot of intermediary programs. And I've made changes in the last couple of days, so it might not be synced up totally. Anyway, assuming that that all works and it is all correct, which it is, consider this pathological situation. When we have many clients that are adding URLs simultaneously, say I've just launched this service as Google's fancy URL shortener, and suddenly a million people try to add URLs at the same time, even though we can safely update the map concurrently, the disk writes might happen simultaneously, like the underlying save calls might happen at the same time. And now some operating systems do writes atomically. And so this might be OK, but some of them don't. And we can't rely on this kind of behavior. And secondly, even if the writes don't collide, it means that every client has to wait for its put to go through to disk before it will return. And we don't mind making clients wait for them in memory map to be updated, because that's a very fast thing to do. But if you're talking about disk writes and a heavily IO loaded system, clients will wait way longer than they need to for their requests to go through. So what we want to do, ideally, is to decouple the put function from the save function and make them happen in separate, not at the same time. So this introduces concurrency. And the fundamental unit of concurrency in Go is the Go routine. And there's a reason why they sound like co-routines, because that's essentially what they are. But a Go routine is a lightweight thread that is managed by the Go runtime. They exist in shared memory, like many familiar sharing models, but they're much cheaper to create than an operating system thread. They have a very small stack. Creating a new Go routine is under about 4K of allocation. So you can create many thousands of them, or even hundreds of thousands of them, if you'd like, without running into too much trouble. So to launch a Go routine, we use a Go statement, which is a similar sort of syntax to the defer statement. It's Go followed by some function call. And when you call Go and some function, the function itself, like the function pointer and the function's arguments are evaluated in the current Go routine, a new Go routine is started, and then that function executes in the new Go routine. Really, 15 minutes left? Oh my god. So we have foo executing in one Go routine and bar executing in the current Go routine. And that's it. That's as simple as Go routines are. But this isn't very helpful. What if you need to talk between these Go routines? And so this is where channels come in. A channel is a conduit, like a UNIX pipe, because it's typed. You create a channel and then give it to two ends of some communication in separate Go routines. And then they can send the things down the channel to each other. Like maps, channels are initialized with the make function. So to make a channel of integers, I call make channel int. And so ch in this case is a new channel of integers. And then communication is expressed using the channel operator, which is a less than dash, like a little arrow. And so to send the integer seven down this channel, we do channel less than dash seven. And then to receive an int from a channel, it's kind of the reverse. We receive a value from ch and then store it in i. The data always travels in the direction of the arrow. So that makes sense. The example of communication between Go routines. Here we have some function main. We create a channel of integers. And then we launch a Go routine calling sum. We pass it some arguments followed by the channel c. And then in the body of sum, we add x and y and send that value down c. And then back in our main function, we receive the value from c and print it. So channel operations, and like they would in this case, they typically block until the other side is ready. So in this case, this send to c will wait until this receive from c is ready to happen. Now, obviously they're going to be ready immediately in this instance. But in other instances, this can be a very powerful synchronization primitive and gives way to some very interesting algorithmic possibilities. But channels can be buffered or unbuffered. And if you make a channel buffered, the send to that channel will not block until the buffer is full. So to make a channel buffered, you simply add a second parameter to make, which is the size of the buffer. So this channel of integers will store 10 integers before it gets full. So instead of making a function call to save, each time we do a put, put can just create a record value and send it down some channel. So I'll add this field to my URL store, save, which is a channel of records. And then when I successfully perform a set, I will send this record containing our can URL down the channel save. And then at the other end of that channel, we need to have something that's actually receiving those values and writing them to disk. Now, we'll create this new method called save loop. And save loop will run in a separate go routine. And what it does is it opens the file, identified by file name, and bails if it can't open it, and then creates a new gob encoder that wraps that file, and then loops forever receiving from s.save, storing the record in R, and then encoding R, writing it to the file, to the underlying file, and logging an error if an error occurs. And so the final thing is we need to modify a new URL store so that it kicks off the save loop in another go routine when the URL store is initialized. And so we create a constant save queue length, which is the internal buffer length of our save channel. And so we can have up to 1,000 URLs being saved before ports will have to wait for the buffer to clear. And then when I create the URL store value, I'll also provide save with the result of making a channel of records of save queue length. And then I call s.load passing in the file name parameter and log the error. And then I kick off with a go routine that's running save loop passing in the file name. So just very briefly, in this program already, there's a lot of magic numbers, a lot of constants. I want to replace them with command line flags. I can import the flag package and set up a bunch of global variables, blah, blah, blah. The one thing I have to be sure to do is call flag.pass at the start of my main function to actually read the command line options and set these global variables to their respective values. And I need to move my initialization of new URL store into the main function so that I can refer to my now-past command line flag. And similarly, we update the listen address and the host name in the ad handler. But I'll quickly demonstrate, or will I? No, I won't. One more thing, the last thing that I'm going to do to this program is so far we have a program that runs in a single process. And while it runs concurrently in that single process, and it will spawn multiple threads to handle blocking operations, so it will always have something to do. But a single process running on a single machine can only go so far. What I'd like to do is turn this into a production-ready web-scale URL shortener. And to do this, I will charge this program into master and slaves. And the way this will work is users will make requests through some sort of load balancer, which I'm not going to worry about now. But they'll hit slave front-ends via HTTP. And these slaves will make RPC calls to the master. And what I'm going to demonstrate now is the implementation of those RPCs and turning this into a slave. So to introduce goes RPC mechanism. It's part of the standard library. And it's very simple. The way it works is given a value of some type. RPC will expose that value, the methods of that value that have this form. And the form is for some type T, they need to have two pointers as their arguments, and only two, and return an error value. And so if you have a type which implements methods of that form, then RPC can serve them over a network. So to make our URL store a RPC service, we need to change the function signatures of our get and put methods. So instead of get taking a key and returning a URL, instead it's going to take pointers to two strings, key and URL, and then return an error. And similarly, put is going to take pointers to URLs and keys and return an error. Now, and once you've done that, you also need to obviously modify the call sites of those methods in our add and redirect handlers so that they do the right things. But I'm not going to show that now because I'm running out of time. But to actually serve our URL store over the network, I'll add this command line flag RPC enabled. And it's called RPC, and it's a boolean value. So if I call my go to with dash RPC, it will serve its get and put methods over RPC over HTTP. And the way we do that is if the RPC enabled flag is set, I call RPC.registerName to register store under the name store. And then I call handleHCTP, and that will hook the RPC engine into the HTTP package. And later on, when I set up my HTTP server, it will also transparently serve RPC through it, using a little bit of magic, but not much. And so now that we have the URL store available as an RPC service, we can build another type, which looks like a URL store, except all it does is pass its requests through to the master server. And so we'll call this the proxy store. And the proxy store is a struct that simply contains an RPC client handle. And so our new proxy store initializer function will take an address. And this is the address of the RPC server. And it will call dialHCTP from the RPC package to connect to that address over TCP. And then if an error occurs, it'll log it. But otherwise, the client will be put in the proxy store field, and that proxy store will be returned to the user. And then we define get and put methods on that proxy store. And what they do is they take keys and URLs, and then they pass through those requests by invoking the call method on the RPC client. And this will call the store.get method at the other end of the RPC connection, which will be another instance of goTo, passing in the key and URL. And so when this returns, it returns the error value from that call. And URL will be updated with the resultant value. And similarly, the same thing will happen with put and key. So this is enough to implement like a pass-through, but there's something missing. The slave has to cache some of this data. Otherwise, all it's doing is adding work. So we already have the perfect data structure in which to cache this data, and that is the URL store. So I'll add a URLs field to my proxy store struct, call it URLs, and then initialize it in my initializer function by invoking new URL store. And the one thing that I'm not showing here is I've just modified URL store so that if you give it an empty file name, it just won't try to read or write it to disk. It still does all the in-memory stuff. It still keeps a map, but it won't write anything to disk. And then I modify my proxy stores get and set methods, or get and put methods, so that first it tries to get from the local URL store cache. And if it finds something in the cache, if the error is nil, I can return. And then this URL value will be updated with the cached value. Otherwise, I try the RPC call. And if the RPC call has a problem, this will return the error. If it doesn't have an error, meaning it's successfully found something, it's successfully gotten the URL from the RPC server, I'll set that in my URL store cache in the local cache. And then return no error. And then the put method is simpler. It attempts to do a port to the master RPC server. And if that's successful, it updates the cache. So now we want to integrate proxy store with the web front end. We want our slaves to serve our web interface and for it to pass all its requests through the proxy store instead of the URL store. Because they both share the same methods, get and put, we can define an interface type called store, which includes the put and get methods. And our global variable store can be of type store. And then in our main area, we can create a master address flag dash master, which is the address of the master RPC server. And if a master address is provided, it means that we must be a slave. If we're being told to connect to a master, this go to instance is a slave. So we instantiate a proxy store pointing to that master address. But if master address isn't supplied, then we must be the master. And so we create a URL store, which reads and writes from this data file. But then the rest of our interface code behaves exactly as before. I've just swapped in an entire new set of functionality. And the interface has been implicitly taken advantage of. And I've not had to do any further changes to the front end for this to work. So I'll just demonstrate this ball in flight. So if I go to slash add on localhost, we'll see we have a URL store running. That's already got a few URLs in there. But then what I've actually written is this little graph, graphing program. And this has got its hooks into my go to instances. And we have a little benchmarking program. And what the benchmarking program is doing is you can see identified by this blue series at the top. The blue series is the benchmarking program making gets against my go to program. And the yellow series down here is the benchmarking program making puts. And the red series is the go to store reporting the number of gets that are being made every second. And the green series is reporting the number of puts that are being made every second. So we can see from this graph that approximately 8 to 10 puts are being made per second by the benchmarking program. And about 100 gets are being made. And they're being serviced one to one. There's only one master and that's all that's happening. What I'll do is just increase the volume a little bit. So we're now serving between 5 and 600 get requests a second and about 50 put requests a second. So we're increasing the size of our data set substantially with every second that goes past. So if I want to introduce a slave, we can see what happens. Introducing a slave, we notice that immediately the red series, the master's get requests have gone to zero. Because now our slave, which is this new purple series, is serving all of the get requests. It's sitting between the benchmarking program and the master. So the benchmarker is hitting it hard. But the number of put requests still remain constant. And they're still being served by the master server because they're being passed straight through. If I add a second slave predictably, we can see that as the number of aggregate gets, the blue series that are being made remains the same, roughly. We can see that each slave, the darker blue and the purple series in the middle, sharing the load about 50-50. And now the master's get requests have gone up a little bit because it has to prime the cases in each of the slaves. Now that each slave is sharing a different part of the load. Adding a third slave, we can see that, again, it's now gone down to about a third. And the master's still sitting pretty low. And the one thing I should have showed you from the beginning of this demonstration is if we look at my process list, these three instances of go-to here are running at just between 15% and 20% CPU. Well, this instance here is running at about 5%. And that's our master. So the master is being shielded from this onslaught of requests. And the benchmarking program itself is kind of doing most of the work. And you notice that I haven't specifically asked for any threads in any of these programs, except my slaves are running about 10 threads a piece and my master is running about 10 threads as well. Well, my benchmarking program actually has 92 threads, so it's hammering pretty hard. And this is now serving like 6,700 requests a second on this puny MacBook Air, which is totally underpowered and pretty much pissed junk. So imagine what it could do on a real server. And this is without having to, this is like genuine, high performance code here, without having to even really think about efficiency. It's all just kind of given to us for free by the fact that Go is such a lightweight language. So while this program works, and you can see it's fairly resilient and it's fairly performant, you'll notice that there's a bunch of stuff that could be a lot better. Obviously, the aesthetics could be greatly improved, could be a lot prettier. Go has a template package that would make this very easy to accomplish. And if you visit the Go website, there's a code lab which goes through creating a web application with a proper interface. The reliability of this system could be a bit better. Currently, if the master-slave RPC connection goes down, it will just die and never be saved. And you could fix this by having some kind of dial ago routine to reconnect every time it goes down and manage the connections. As the size of the URL database grows, it might not fit in RAM anymore, so we might have to shard it out and do something a bit more complicated. And if you wanted to support deletion of URLs, that would complicate the interactions between the masters and the slaves, because the master would have to report that URLs have been deleted. But these are all exercises for the reader. And this code is all available in my GitHub repository, so you can check it out. There's a lot of Go resources. If you're interested in learning Go, the first place to go is golang.org, where there's lots of documentation tutorials. And there's even, if I can demonstrate very briefly, there is even a box on the front page of the website where you can type Go code and have it compiled and run on one of our servers. There's our Go blog and there's a bunch of other stuff. So does anybody have any questions? Yeah. Hi. What I haven't heard in your speech was why would I have to use Go and not, for instance, Java or C++ for this? Well, if you like the syntax of Java and C++, then have fun. But honestly, once you get to a certain size in substantial, like the types of programming approaches that Java and C++ encourage is a very tight, blatant, hierarchical approach to running software, which once you get to a certain size, it actually starts to really weigh down on you and has a much more compositional, lightweight approach to running software. So I mean, it's a long conversation if you want to get me afterwards. Any other questions? So the two things that turned me off of Go when I first saw it was, first that it has null implicitly in the type system, i.e. if you specify a function that takes a string, it either takes a string or it takes null, and the same thing if you return it. That's not actually true. There are certain types in Go that can be null, and a string is not one of them. Only a reference type can be null in Go, so if you have a string or an integer or a float or any sort of concrete data type, it can't be null. It can only be at zero value, which is usually numeric zero or an empty string. But if you have a pointer type, a pointer can be null. Or if you have a map type, a map can be null if it's not initialized. Or if you have a slice type, rather, a slice can be null. So does that mean that I can write, that I can implement a type that cannot be null? Yes. And if you have a struct type, you can use a struct value. And a struct value cannot be null. If you need further clarification, I can talk about it after. Yeah, one of the things that strikes me is that Go is apparently a language which is designed for concurrency. Yet when you can compare it with well-designed concurrent languages, it becomes clear that Go's concurrency primitives are, at best, half-arsen, at worst, actively dangerous. You have designed a concurrent language which is based around shared mutual state. And that's pretty dim-witted, if I may say so, to begin with. And then you also have channels which block callers. You have core primitives which are not type safe. I guess my question is how you were able to design a concurrent language which is so terrible at being a concurrent language? It would be nice if you could have phrased your question in a way that wasn't incredibly insulting. Because I do think that you actually have a point. But what you're actually pointing to is a philosophical difference. Go is a language that is very close to the machine. It has its heritage in C. But we allow you to shoot yourself in the foot if that's what you want to do. It's not a immutable sort of fenced, non-shared memory model for concurrency. Because that's not the way processors work on the machines that we write code for. So Go is more of a pragmatic language in that sense. But Go is primitives like Go routines and channels. They actually allow you to do what we say is that you share memory by communicating about memory instead of communicating by sharing memory. So while we allow you to use things like mutexes to implement traditional, evil shared memory concurrency, channels actually allow you to do in a way that is very efficient and easy to understand and predictable, like pass around a pointer to a region of memory. And when you write, it's very easy to write code with channels that passes pointers and passes references around that is safe. Because you can see there on the page that it's safe. When you send that pointer down a channel, don't use it anymore. And then it's safe. So yes, it is possible to totally screw it up. But the tools that we give you actually make it very, very easy to write safe, reliable, concurrent code. OK, let's thank the speaker and have a lunch break.