 All right, so I'm here to talk to you today about API design. So why? Why am I here to talk to you about API design? So how many out there, this is an easy question. I'm going to throw some softballs at you here. How many of you write software for a living? OK, I'm hoping quite a few of you. Great. And how many of you design APIs for a living? OK, see, all of your hands should be going up right now. If you write software, you design APIs for a living. You just may not know it yet. And that's one of the reasons why I think it's important to talk about it. And it's actually not discussed very often. I don't know. I mean, it may be discussed, but there's very little written about good API design. I suppose that we're supposed to just know good API design after we've been developers for a long time. But I think we can do better than that. So my talk today is to get us not only thinking, but hopefully actually some of you go and start writing up things about API design and start really delving a little bit deeper into it. So APIs. As a software developer, APIs are your product. This is both internally and externally. So for example, you have APIs. I mean, anybody who's developing web apps these days, probably at some point either already has an API, an external-facing API, or will eventually build one. But also internally inside of your apps where your apps talk to each other or even your classes talk to each other, those are all internal APIs. So there's APIs all over the place. And those are ultimately the product that you're developing while you're writing software. The other reason why it's important to talk about APIs is because they outlive implementations. So anybody who's used a library that has evolved over time will notice that the APIs often, they'll be improved. There's going to be some stuff that's deprecated. But the implementation behind them is going to change all the time. Getting the APIs right, though, that's an important part of getting a library that's really useful. So it's important to talk about it because they outlive the implementations and because they lead everything else. So essentially, a good API design is going to lead a lot of other things like implementation at the client level. So that's another reason that it's important to talk about it. So a few principles here, some things that come from some of what's already been written about API design. I'll talk a little bit about some of the presentations and papers that have been written about it. There's really not a lot, but I'll give you what I can. The first thing that's common in these is that any API should be easy to learn. API should be designed around the principle of least astonishment. If you think it should work some way, when it does, you're very happy. When it works the way you expect it, you're very happy. So the principle of least astonishment is a very important in API design. The other thing is that what makes something easy to learn is when you think about naming. So you think about naming in the context of your audience and who's going to use the API and who's going to talk about the API. So the other thing that another principle that's good to follow for good API design is a well-designed API should produce readable code. So that's from, for example, client usage perspective. So if you have a class that's talking to another class inside of your application, if they have a good API between those two classes, then the client code is going to be pretty easy to understand. It's actually probably going to be pretty easy to write. I'm not going to say it's going to write itself, but it might get pretty close to it. So producing readable code is a good sign of readable client code is a good sign of good API design. Another principle of good APIs is that they are easy to extend. So what this means is that they should grow with the requirements. And they should also be designed with this notion of growing with the requirements in mind. Because APIs are going to, as I mentioned earlier, they're going to live for a long time. The API is not going to be something you're going to put out, and a couple of weeks later, you're going to change it. Because then nobody can write a client to it once that's out there. So you should be designing APIs with growth in mind. They should also be easy to split into smaller APIs. Because as APIs grow, they become more complex. We were hearing that earlier from Michael. He was talking about complexity of software. Over time, things become more complex. It's just the nature of software growth. But the better API design. So it makes it easy to take one API and break it into multiple APIs. It might be breaking into two, three, grouping together ideas as the one idea starts to meld into multiple ideas. It's too complex, so you can break it up into other parts. So being able to extend and to break that API part is an important part of good API design. Another element that's important is that it should be hard to misuse. A good API is something that is really hard to write bad code to. It goes back to writing good code. You shouldn't be able to abuse that particular code. You should be able to write correct code. And that's a bit nebulous there as a term. But essentially, your code should be able to consume that API. And it should not be your client code shouldn't be complex. A good API that's designed that's hard to misuse avoids relying on side effects and order. So a bad API design might rely, for example, on the order of some state that you've set up in advance. And if that order is different, then your API behaves different. That type of thing is a bad API design smell, essentially. So a good API design is going to minimize side effects and minimize dependency on things like ordering. That also means, for example, calls. Let's say that your API, to do something, you have to do three calls. If changing the order of those three calls results in things breaking, that's a good example of a bad API, because it's coupled to the ordering of the calls. And the final principle from this first set of principles is it should be sufficiently powerful. So a well-designed API should be as small as possible, but never smaller. It should only implement the functionality required to get the job done. You shouldn't be going above and beyond and adding functionality because, oh, well, we might need this in the future. Or, oh, this is actually a really fun. This is a challenging problem. So I'm going to add these extra calls in here that somebody might use in the future. No, it really should only implement what is necessary to get the job done. A good thing to remember is that adding to an API is really easy. It's easy to add a new method to an API. Removing a method from an API is much more difficult, because all of a sudden you're breaking dependencies. And so keep that in mind. That's one of the reasons why you only implement what you absolutely need. Because if you ever have to remove it, things are going to get really hard really quickly. So those are the basic set of principles. And that comes from papers like a presentation by Joshua Block from quite some time ago. He was speaking about in Java in particular, but there are other examples as well. There was an example. There's a really good, it's the little book of API design or the little paper. I can't remember the exact title. Show it later. But it's neat. It talks about QT and the development of QT at Trolltech that was acquired by Noki. And it talks about their evolution of API design. So those principles actually are common across that. What I'd like to do is I would like to reword them and hopefully give you something that might be a little bit easier to remember. And so I'm going to go with the letter C here. The first word I'm going to talk about here is consistent. So good APIs are consistent with existing APIs. If you use a particular style of writing, so if your methods follow a particular style or your API structure follows a particular style and you have another API that's related to that, in an ideal situation, those should be consistent. Because when they're inconsistent, developers get confused. They're like, oh, well it worked over here this way, but all of a sudden it works very different over here. So consistency is important. It same applies for naming. If you have the same concept used in different places and all of a sudden you name those two different things, so let's say, for example, arguments to a method. In one case you might name something. One thing in another, it's the same thing, but you name it differently, you're going to cause cognitive issues there with the developers who are going to use it. They're going to get confused. So there's a term called economy of concepts. So that is introduce new concepts in a frugal fashion. And what I mean by that is that APIs are essentially a way to describe the concepts of whatever the business need is, right? You're conceptualizing that you're actually turning into code. So be frugal with that. Don't introduce new concepts into APIs if you don't need to. Clear, that's the second one. That means code should be clear. So clarity of code and clarity of intent, all right? So in other words, whatever that method signature is, it should be pretty damn clear what it's doing, all right? Because otherwise, again, you go back to confusing the clients, the people that are actually going to write code to it. Should be convenient. It should be a lot easier to write code with your API than to implement it yourself or to use some other API. It's important to make it easy. You want your code, your libraries to be really easy to integrate into whatever is being used. And the best way to do that is instead of making things complex, think in terms of usability. Think of your API as something that people are actually going to use. And so try to keep it simple as opposed to complex. Actually, there's a Tom Preston Warner talks about this, about readme-driven development. I don't know if you've heard him do this talk before, but it's essentially write your readme first as if you were gonna use that code. And that actually will drive things like convenience because you'll naturally be the first client. You go, oh, what's the most convenient way for me to do this thing? Okay, what's the least number of methods or what's the most clear and concise way of writing the methods? So, conciseness, so be brief. Brief APIs are good. You wanna be just comprehensive enough to actually cover the needs. And then finally, it should be complete. Again, this goes back to, it should do the job no more, no less. So those are the Cs that I was talking about that I wanna sort of reiterate here. Be consistent, be clear, be convenient, be concise, and be complete. So, now we get the fun job of taking a look at some examples. So, I'm gonna use Ruby. I know, it's shocking. I'm gonna use Ruby and I'm gonna use probably the prime example that the Ruby community, well, I think is one of the best examples of API that has some questionable design in it. Net HTTP, the standard library from hell. I didn't know there's some people that backed this thing up as saying it's okay. It's really not, and we're gonna go through now. While I'm going through this, I want you to ask yourself the question about those five Cs. Is this thing consistent, clear, convenient, concise, and complete? Okay, that's what I want you to ask yourself while I'm doing that. All right, so let's look at four different calls here, all right? So, when you're looking at this, we've got get print and it passes a string, actually two strings, and then we have get print and it passes a URI, and then we have get and it passes two strings, and then we have get and it passes a URI. All right, the only difference here between get print and get is that get print has the side effect of writing to standard IO. Standard out, basically, all right? And you'll notice here that we've got two different signatures, but that first signature, the one with the host name and the path, leaves a lot to be desired in both cases. I mean, what scheme are you using? Right, what port are you on? How do you set those options? And then you can use a URI parse, but why can't I just pass in the string, hb colon slash lastexample.com and you parse the URI for me? This seems reasonable, right? So moving on, here's another example. This is some good code here. All right, so we're gonna start on this host and then we're gonna do a get on slash. It seems simple enough, right? But let's take a look at the signature for get. Okay, it's got three arguments, two which are optional. The path, that makes sense, right? But what are those other two? Okay, so a knit header is a hash of request headers. Why a knit header? Why not just headers? I don't know. Okay, so it doesn't make any sense. Well, the second argument, though, is even better, the desk argument. The documentation says the desk argument is obsolete. It still works, but you must not use it. That's the actual documentation. Okay, and oh, by the way, it gets better. It gets better here, okay? So, this is some more documentation inside. There is called with a block. Yields each fragment of the entity body in turn as they string as it is read from the socket. Note that in this case, the return response object will not contain a parentheses meaningful body. So what will it contain? You have no idea, you just know it's not meaningful. All right? Okay, wait a minute. Except in version one one, in which case this method might raise an exception for a 300 redirects. And in this case, you can get a response object by an exception response. Okay, so there's a lot going on here. And by the way, we're not done because in version 1.2, this method never raises an exception. Okay, so that's one method. That's that get method. And all you have to do is go read through the API. There is tons of this stuff. Jeff, this slide's for you, baby. All right, so the get print, there's side effects, there's unnecessary methods. It does more than it should. So you don't, you have a separation of concerns. Start, there's start as a class method. There's start as an instance method. So there's unnecessary duplication there. They have post, and then they have a method called post2. Do you know what they do different? I have no idea. I'm sure it's somebody knows. There are classes in there that are just sprinkled in as if they are, they're, I don't know even what they're, there's like a class called D. I'm not, I know what it does. It's for debugging, but it's called D. And then they have a proxy, which is a method that actually is a class. It's very confusing. Version one underscore one, you guys can't see this because it would just be too painful to show it to you. This is how you actually turn it to version one one of the HB protocol. There's side effects. There are, there's a new object method, new OBJ, completely unnecessary. All it does is call new. So I mean, it's just overhead. It's why is it even there? There are exceptions for flow control. Abdi, you're gonna love this one. Using exceptions for 300 redirects. I mean, of course, who doesn't use exceptions for flow controls? People who design good APIs, perhaps. There is, there's SSL methods. So this is again, we're back to lack of separation of concerns. So this library handles SSL just rather poorly. Okay, so it does that. There are proxy methods. It handles HB proxy again, not too well. So it's just, it's a really great example of a horrendous API. In all fairness, it's been around for a while and it does, you can get the job done. It's just really complex. So let's take a look at some examples of things that are maybe a better way of doing HTTP, okay? So these are examples I think that are better APIs. And again, ask yourself, are they consistent, clear, convenient, concise and complete? So Typhus, this is a, an HTTP library. So here's an example of a Typhus. You actually construct a request. You pass in the full URL as a string. Typhus is designed to do multi, multi, so parallel request management. So you actually have this, this thing that you queue up the request in because you could queue up a bunch of requests and run them in parallel. And then from the request, you get the response and you get the response body. It's pretty clear. But the nice thing is if you have, if you need a shortcut, it's as simple as this. Create a response and ask for the response body. So one thing that's good about API, and we love, in Ruby, we like shortcuts. I mean, it's nice to have methods that are convenient. This goes back to the convenience. This is, I think, a good example of convenience. I think we could go further and we'll see that in just a minute, perhaps. But this is an example, I think, of a better way of calling HTTP. Here's an example if you wanna do a post. So instead, you send the params as a hash, nested hash. And again, you get the response body. It's consistent. All right, it's fairly consistent. So another library that does this, too, is Faraday. Faraday is also another HTTP library. Again, ask yourself, is it consistent, clear, convenient, concise, and complete? So it looks pretty familiar. So they use essentially the same basic pattern for being invoking a get HTTP. This actually is simpler because I don't have to create a request object. I just have a class level get method that I can call. But the nice thing is I can also do things like this. If I want to, and the way Faraday works is it actually lets you inject modules kind of like how Rack does, but for HTTP requests. And so I can say, okay, I wanna build her that is gonna be my connection. It's gonna be a JSON, it's gonna have a logger in it. And then I can call connection get. I could pass in headers if I want to. I don't have to, those are optional. And again, I would call response body. So it's consistent. You can also, I think, open URI, I would argue that that's a pretty good API design as well. Because sometimes the best API is no API. And that's kind of how open URI works. Open URI, you say open a URL, and it treats it like a file. So for basic usage, I think it's a really clean way because it's very comfortable in Rubyland. It's like opening a file. You can go through each line. I mean, you're treating it just like a file. So in this case, they didn't create a new API. They just reused an existing API in a way that is pretty intuitive if you're a Ruby developer. So again, that's awesome for you, Jeff. I love it, man. Okay, so again, you should be asking yourself, an API design, is it consistent, clear, convenient, concise, and complete? There's no, I couldn't get a unicorn, man. I can't only take it so far. Okay, techniques. So let's talk about techniques for actually building better APIs, for designing better APIs. The first technique, I know this is gonna shock you, but you actually have to ask and talk to the people that are gonna use your API and think about it a little bit, okay? So talk to the people that need it. Know what they actually need, okay? And then be the client. Actually put yourself in their shoes. It gets back to things like developing, whether it's read-me-driven development, or, and you're gonna be a combination of things, TDD or BDD as well. Basically design it so you're using the API while you're designing the API, all right? That's the first thing to do. Think about your clients first, okay? And help them create clear and concise requirements. I'm not talking about requirements, documents that are 30 pages long to get blessing by some middle management. That's not what I'm talking about. I'm talking about sitting down for an hour and saying how are you gonna use this API? Let's talk about it a little bit, all right? Write some sample client code, okay? And actually have it running it, run it every once in a while against your API and see where the problems are. So tease out the bad parts of your API. Think about your implementation second. The client comes first, your implementation comes second because as I said before, APIs last a long time. Implementations change all the time because they're behind the scenes. So favor a clear and concise API with a difficult implementation over allowing the implementation to bleed through the API. This implementation bleeding through the API can end up in usually pretty horrible client code. And actually this is one of the principal ideas behind Ruby. Ruby is why do you love writing Ruby code? It's because as a developer, it was like it was designed for you in mind. It wasn't designed for the compiler, it was designed for you. So write your client APIs the same way. Address your audience. Define APIs with the appropriate level of jargon. And I'm gonna say this jargon is actually useful if the people that are using something speak that jargon because it allows you to shortcut some of the discussions because you can use terms that are common to that particular business area. So define your APIs with the appropriate level of jargon for the type of consumers. Domain experts are gonna expect one thing. Non-domain experts, all they'd want to do is just not be baffled. They want something that's simple. So and you can combine those two but you have to, I mean that's tough. You have to, a lot of it's writing good documentation as well and things like that. Choose good defaults. Be sure to document those defaults. Convention over configuration is a good thing but you also have to document why you choose the convention and then allow it to be configured later because there will be people that are gonna go around your conventions and they're gonna need to. Separate your API and your SPI. So SPI is the service provider interface and the API is the application provider interface. Consider the API, the public consumer interface and the SPI being an internal public provider interface and you can use to extend the library and things like that. There's a whole another talk I'm sure. We could talk a lot about SPIs and the fact that in the Ruby community we don't really think in terms of APIs and SPIs right now because our language is so malleable but there might be some benefit in thinking in those terms. And again, ask yourself as you're writing the API, is it consistent, clear, convenient, concise and complete? And remember, Jasmine Blanchett, the little manual of API design, that's what the little PDF said sometimes the best API is no API. So think about how you can use existing APIs or existing patterns that people are already comfortable with to implement an API. All right, so some things for you guys to think about as I close out my talk here. So remember that if you're a software developer then you are an API designer and you need to design your APIs with purpose and intent. It's not good enough to just do the first thing that comes to you. You actually have to have purpose and intent when you're doing it. So the questions I want you to ask yourself, I think I found this a useful exercise, a couple questions. What are some other examples of good APIs to learn from? Go out and search out some, and you don't even have to search. Just look through the stuff that you're probably using all the time. Most of us if we're writing code that we're probably relying on at least a few other libraries both internally and externally, maybe quite more than a few. So look at them and find the good APIs and learn from them, right? In Rubyland, what can we learn from APIs in functional languages? I mean I mentioned how side effects and ordering and things like that can be dangerous. What can we take out of functional languages that can teach us more about good API design? And then the final thing to think about is how can you refactor your APIs to make them better? And you have to keep in mind the challenges that you'll face for your existing users as you're refactoring. So you're refactoring implementation might be fairly easy as long as you don't bleed it through. But how do you actually refactor the public facing API? Something to think about. Those are links to all the photos that I used thanks to the people on Flickr who were so gracious to make them creative comments. The two things that I was mentioning is one is Joshua Block, how to design a good API and why it matters. This is a fairly long slide show and then the little manual of API design which is pretty neat as well. And that my friends is it. I want to say here real quick, I think I'm gonna go ahead and show. I have chocolate down here. This is one of the things I do. I come to conferences and I bring chocolate. I live over in Europe. So please, I'm gonna leave the chocolates up here at the front of the stage. At some point come grab a chocolate and if they run out I'll put a few more up there. We have five minutes if we wanna do questions. If anybody has any questions now would be the time to ask. Yes, down front. Okay, so how can I stand up here and say if you write code for a living that you write APIs? The reason is because all code is stuff talking to each other. The vast majority of code is communication between entities, between things, or functions that are calling other functions and things like that. And so you're using your own code internally. You're using your own APIs. And what I mean is that even if you're not developing public APIs every method that you declare on a class that is public is now an API. Every method that is private is now an internal implementation. That's important to understand because if you just say, and there's discussions in the Ruby community about well what should be public and what should be private? In all fairness you could write everything public and then only document what is actually an API. But the truth of the matter is I find that the best way to describe what is your API on a particular class, because every class the methods are the API, right? Is if it's a public API it's a public method. Everything else should be protected or private. And it's not because they can't get to it, it's because it's a signal that indicates to them hey, you really shouldn't be using this because we're probably gonna change it. Or it's protected therefore it says hey, if you subclass you can use it. But otherwise don't because we could break it later. So that's why I say that is because all code be right, everything has public methods or public functions and that is the API. Other questions? Yes, what's a good way to get consensus on what should be a standard API in this particular case for the Ruby standard library but I think the question applies outside of that as well. I'm a firm believer that you're, all right I know standard libraries are great because it gives some people a starting point that comes with the language. On the other hand I think standard library should be as small as possible, okay? And I think that essentially you should have just enough to get some things done but I think that you should be pulling in libraries from other sources. We should be talking about a smaller standard library in my opinion, not a larger standard library. That's my opinion. So that's the answer to the question. To get consensus of rewriting some of that stuff that's just somebody out there is gonna go do it and the answer should be then at Ruby, like major version we will remove these standard libraries end of story. And it's a call from the top. It's like we will be having major version, it will be backwards incompatible that is the way it is. Yes, Wayne? How should you version your APIs for change? Well, external, sorry, the question is how do you version your APIs? What was your comment? What is it? Oh, semantic versioning, semver. So semantic versioning is the suggestion how to do APIs. Honestly don't know as far as internal class APIs in versioning them inside of between your classes. It's one thing to talk external APIs, arrestful APIs and things like that. I think internal APIs, I don't know, yes, semantic version at the library level but can you actually, right now for example, if you're gonna deprecate something, what's the best way to deprecate? There's still a lot of open questions. So I quite frankly don't have a versioning scheme that I think is the best for internal methods. It's also very possible. The question is how do we, what's the best way to do it in web APIs? And I'm gonna save that for another talk. I'm not gonna answer right now because there's lots of ways to do it and it takes way more than the two minutes that I have, one minute I have left. There was a question over here, yes? It's an SPI. Okay, SPI, well the question is what is an SPI? SPI is a service provider interface. So let's take an example. Let's say that you have a public API that is used for getting context out of an address book. That's something simple, right? So who might provide context, contacts, that's really, sorry, not context, contacts. Who might provide contacts? Well, you might have an internal database but you also might hook into the system's address book. So for example, let's say OSX, you might hook into that. Or you might hook to an external RESTful API. All of those are the services that provide the actual object. So writing an SPI is defining a common interface for those services to plug in behind your public interface. That's what an SPI is. And I have time for one more question if somebody wants to ask real quick. Otherwise I will wrap it up. All right, thank you everyone. Again, remember there's chocolate down here. Please enjoy it and have fun at the conference.