 If you didn't already know, you're in Building Cooperative APIs with HTTP2. I wanted to quickly just give a shout out for a couple of Drupal days. It's going to be in New York City on August 17th through 19th. It's a great place to come, share experiences about building decoupled sites, learn from one another, ask questions, voice concerns, just collaborate on some really cool and awesome stuff that's pushing the boundaries of what Drupal can do. Kind of bringing us into the future of what we can do with JavaScript with Drupal. Yeah, so try and make it decoupled Drupal days. I'm Gabe Solis. I'm a senior engineer at Acquia. Jason API Co-Maintenor. I really love HTTP2. I really love reading specifications, RFCs, and I do sleep over APIs. To figure out how we're going to build cooperative HTTP APIs with HTTP2, we kind of have to understand where we're coming from and some of the concepts that are underneath HTTP, so therefore why HTTP2 is different than HTTP1, and then also some just basic concepts about how we build APIs in general and how they work, how client and server can work together. At the lowest level, that's as low as we're going to go, it's IP or the internet protocol, and it's very dumb system. It's just you've got two nodes, two computer systems communicating with one another, sending data back and forth between a source and a host, and they find each other on the big, wide open internet with IP addresses. Very raw, just bits and over the wire. So the internet protocol describes how those two systems can communicate. And then on top of that, people have what's called TCP, and that's the transport control protocol. And TCP is a way, a mechanism for communicating over IP, but it adds specific features to IP that make it a little bit easier to use, right? So it's reliable, ordered, and air checked. Reliable meaning if I send a raw packet of data over the wire, I can be sure that it's going to end up in the client. I don't have to account for some packets of information not arriving, or just missing packets altogether. Ordered makes it easier to understand, because if I send like a bunch of JSON over the wire, that first half is going to get there first, and the second half is going to get there second. Because there are lots of different ways that that data could go over the big open internet, and so they might arrive out of order, but TCP means I can be sure that they'll be there in the order I sent them. And then air checked just means nothing got corrupted along the way. And so HTTP then builds on top of that protocol. And HTTP stands for Hypertext Transfer Protocol. And there's one distinct feature for what we've seen so far about protocols, and that it's an application protocol. So it means it's actually for being applied to real systems that you're going to build. It starts to get into actually how to build things that are what we consider applications, right? Rather than just infrastructure underneath. And another thing to call out is hypermedia, and that's going to be really important later, because HTTP is intimately intertwined with the idea of hypermedia. It's kind of what makes the internet over IP the web, right? The worldwide web, that's hypermedia. So what is HTTP One? What is what we've been using for over 20 years? And we all just kind of take it for granted, but really underneath the hood, just a quick reminder, we're going to look at kind of how it's put together. So at the highest level, there's a few basic concepts. That's URIs, Methods, and Messages. A URI, it's a Uniform Resource Identifier. We typically always see it as URL. It doesn't necessarily need to be one. What makes a URL different is that it's locatable, and that kind of means like, if I have example.com, I can use DNS to find out that I'm supposed to go to 192.168, blah, blah, blah. But you could have a Uniform Resource Identifier that doesn't have a DNS resolution. It's not that important. But then we see it as like this HTTPexample.com, EPIUserOne. It's a Uniform Unique Identifier for the user with ID1. That's where the user is, okay? Breaking it down, there's Scheme, Host, Port, Path, and Query. That's just the basic components of it. And so then HTTP has the concept of Methods, right? And you've maybe seen these before if you've built APIs or if you worked with REST, those are Git, Past, Post, Patch, Put, and Delete. There are others, but what they do is they describe actions between the client and the server. Things that you want to change, right? There are a couple others. Head Options, Trace, Connect. But they're really more about discovery. They're not about applications. Options kind of says, what am I allowed to do? Git says, get me the resource at a particular URI. Post says, create a resource at this URI. So maybe add a new user. Past, Post, Patch, Put, Delete. All different variations of that idea. And then statuses are kind of from the other side. So if you've got a client and a server working together, Methods are what the client sends and statuses are what the server sends back. And so they categorize basically errors or information. They categorize that communication between the client and the server. 200 level errors, they basically say, I understood the message that you sent me. I did what you expected me to do. We're all good. And then there are subcategories in there, like 202, 201, 204. They're all basically saying, I understood and we're good to go. 300 level errors, they say like, I understand what you're asking for, client, but maybe you actually meant this. You should be talking to a different resource. 400 is just like, client, you're trying to go home. It's a bad request. It's the client's fault. And 500 is kind of like, it's not you, it's me. I'm sorry, whatever you did was okay, but I really screwed up, you know. And so we talked about that idea that there's a client and server and they're sending messages back and forth. And so that's the fundamental idea of HTTP is that you have requests and responses. And they're, they basically start with a line. They're composed of a URI method, status headers, which are meta information about the message and the body. So what's actually there? And then headers, like I said, describe the resource. My arrow's key just stopped working. There we go. And so for example, an HTTP request or a response, it's going to say content type might be the header and it's text HTML. And then of course the body follows and that kind of tells the client how to interpret the information that's being sent. Don't try and parse this as JSON, parse it as HTML. Alternatively, you could say it's JSON and so then you wouldn't be looking for angle brackets in HTML. You're looking for JSON. You're looking for a co-operation between the client and server there. Just looking at a request. It could be very simple with no content in it. You say get, you know, give me the user one and then the response all put together. Here you see the 200 okay, blah, blah, blah. So then on top of that, right, we start building APIs and we start talking about REST. And that's kind of like the lingua for APIs built on top of HTTP. And what does that stand for? Sometimes we just hear like something is RESTful, right? And you might think like, okay, that means it's going to be in JSON and it's probably going to be some different URLs that are going to be generally structured like some type, like a user or an article or something slash ID. And I know I can share those between a client and a server. REST stands for reference and patient state transfer and it's important to understand that it's really about moving state back and forth between the client and server. It's not about performing actions. It's not about behavior changes. So if you're familiar with Drupal, like you can't clear cache in a RESTful way because you're just, you're telling Drupal to do something but you're not transferring state back and forth, right? And so REST is not actually a protocol. We've been talking about IP, TCP, HTTP. REST isn't a protocol. Really it's just kind of like a style of building things on top of that application layer, that HTTP. So it's just principles about how you might construct an application. And it's totally, it's always HTTP. So what are those principles? They seem rather obvious but they end up having really like huge ramifications for how you design a system that allow a client and a server to evolve over time and be really robust and to develop independently of one another. So if a server needs to change out its database and switch from MySQL to Postgres it can do that or if you need to go from Drupal 7 to Drupal 8 and still keep operating you can upgrade that without necessarily updating like a JavaScript client. That's kind of the promise of decoupled. Because what that client and server does is it creates a separation of concerns, right? The client is going to handle presentation of the data and the server is going to handle storage and maybe some business logic like what to do when a new user is created. Maybe you need to send an email or something like that. So if you've been hearing about decoupled sites that comes from this idea of RESTful systems being client and server well they influence one another rather. Stateless is the idea that that HTTP request and response have to be independent from one another, right? If you send two or three requests they can't affect each other in any way, right? So the way you might think about this is if I send a request and I say get some resource and I get a 403 back it's denied to me I don't have permission to view that resource and then I go to a login page and I do some login there and then I go back and I retry the same request if I didn't change anything about that first request it should still be denied, right? It shouldn't be that I made the login call I actually have to change something about the request so that might mean putting a cookie header on there or an authorization header that's what makes it self-contained it's independent of one another. And then cashability is the idea that your resources the representations that come for it have to share information about if they can be stored for any given period of time so if you've got a CDN how long can it cash this response or if you're trying to build an offline app how long is this resource valid or something like that? And the reason you want things to be cashable or so you can eliminate requests and responses altogether because that always has a cost, right? So if you make a request to a server even though it's really fast over fiber optic cable you're never going to really get it under 100 milliseconds if you're going between the US and Europe just unavoidable physics there's time that it takes to make those two that cycle of a request and a response so if it's cashable you can kind of eliminate those requests altogether or maybe cash something on the edge and make that latency lower a layered system is that if a system is restful it should be able to have layers put in between like proxies or load balancer or CDNs and nobody should be the wiser the server that's generating the actual resource and the client they don't need to know that something was in between them code on demand is kind of an under underrepresented or people don't really think of it very much but it is part of rest and it's optional but it means that you can actually send code in a restful way that tells the client how to understand a resource you might think about that in terms of like doing client side validation if you wanted to send a schema for a user and you wanted to be able to say certain rules like you can't have these combinations of fields or something you might be able to ship some JavaScript over there that can run that validation it came about for making like Java applets and stuff but it kind of fell out of favor and the uniform interface I promise for wrapping this stuff about rest up it's that kind of like what I was saying you could switch between Postgres and MySQL representations are decoupled from their storage and so when you're operating over a restful API you're not just like sending a SQL query you're not saying like if I want to post a user you don't actually put in your request like insert into table you know values you're not having to worry about the actual implementation details so it allows you to build things in the couple of teams and version things and progress forward and then of course there's the idea that you use hypermedia to communicate state so hypermedia is this idea is linking and that's how you can make represent the state on the server and the client and we'll look at that later but like if you're getting a collection of resources and you want to know is there another page you don't necessarily have to say there are five pages for a listing you just put a link in there to the next one and that communicates the self the fact that there's a link there that there is a next page so hypermedia kind of sounds a little fancy but it's really quite a simple concept if we just think about text that's not hypertext it's just like this right I'm a teapot short and stout and all of a sudden we've made it hypertext all we did is we added a link to text and that link is saying this bit of content can reach out and it's referencing something else it adds information to just something simpler and hypermedia it's just the difference between saying hypertext or hyperjson or hyperxml hypermedia encompasses them all and so you can have links in json and they can be represented in different ways but those links can exist and so you can have the same exact representation in json as you do here you've got the line of text and then something else that it references json you have the same content and then a link out to its explanation and that hypermedia creates kind of the web as we know it you can think of everything on the web as these nodes with lines in between them edges in a graph and they create this web of information that inform one another but also describe the relationships between different resources so the relationship between a listing of users and a particular user so thinking about a very traditional website you might have the home page it lives at index HTML and so a client might go and request that it downloads the HTML and it notices that there's an image tag with a link, a source, right, to a hero and the browser is going to download that it's going to notice that there's a style.css and it's going to grab that and then when it gets the style of the CSS it's going to see that there are probably icons in that CSS as well or fonts or things like that and it makes up the whole page it's a big graph or tree of information that has to be fetched to represent that thing, that home page resource all at once but those hypermedia links don't just make up a single page they can represent whole applications so if you think about a website in general it's just the home page there can be the home page that links to an about page that also links to a post page and that post page also links to individual posts and so not just one page but a whole suite of a whole application lives under that idea of linked hypermedia then we see something that's a little bit more restful it's posts, post two, post one and those posts might reference tags and it makes up that graph so if you're an API client or something you're getting posts and then you want to learn more about the second post you might traverse that link download that other data and of course the graph then changes so that now that you're at post two or previously we were at just posts at the head of that graph what you can see changes so now you've got tags and author tags one and that graph is always expanding out in front of you as you move along so in HTTP one how do we understand this whole graph at once we have to make requests we send a get request for index data HTML we send gets for heroes and styles and so what is the actual order of operations when we send those request responses we first get index.html and then we have to download that HTML and we notice okay there's some style.css let's go get that then we're going to go get the hero image and then of course we finally we've received the style and so now we see we've got to get the separator and we've got to get the list bullet and so of course that's really slow you make these requests and they're sequentially operated and so to download a whole page with maybe hundreds of resources that can take a lot of time because you just have to do one after the other after the other and so people started trying to optimize that system and so browsers basically looked at TCP and they looked at HTTP and they said okay how can we make this a little bit faster first we'll get the HTML but then we're just going to open up two TCP connections and we'll do it at the same time and we'll ask for the style and the hero at relatively the same time and that makes everything a little bit faster we download the styles and then we ask for those icons as soon as we have the CSS and of course it starts to get optimized we're bringing that waterfall down it's becoming faster and faster and so that encouraged people to do strange things like well we still don't want to make multiple requests for all the icons let's put them all into one big bucket and download the whole thing at once and HTTP2 kind of said well maybe we don't need to have this big waterfall of content when somebody asks for the index.html let's just send them everything at once so HTTP2 introduced this idea of server push and that's the big thing that is supposed to make it so much more performant the idea is you get that index.html and the server already knows you're going to ask for the style.css and the hero image and all the icons and so it can just send it to you all at once and so people have been saying okay this overthrows that idea of maybe doing spriting with your icons or using link preloaders prefetchers and all this kind of stuff that of course introduces this idea of the server is kind of forcing you to download things that you may not need and so that's the goal we're trying to get that waterfall smaller and smaller so we've talked about HTTP1, we've talked about REST we've talked about these basic principles and now I kind of want to talk about cooperation and what allows clients and servers to cooperate and that's through specifications specifications impart meaning to a lower level protocol so HTTP imparted meaning to TCP it says if I send a request a packet of information and it has the word get in the top means you're supposed to give me something 200 statuses mean it worked, headers describe the body blah blah blah specifications allow you to take three minutes okay wow, alright alright so schema lets you understand the data types that are there and haydoss links lets you say like are there more pages, can I delete this does this cart have products can I make a withdrawal if a link that exists there to withdraw content or withdraw make a withdrawal that link doesn't exist if the bank account is empty so we can look at can everybody read this cool we look at a document that might be at like APA slash content and we see the next link is there so there must be more pages create is here, there's a link there so we must be allowed to create content I'm just going to skip very far ahead so lets say you're an api thing basically a server can't push all the information you need it doesn't know what the client needs if you're in a decoupled world it doesn't know that if you're loading a certain post that you also need the comments and if you're a decoupled client you might be getting a certain post and say I don't know it's just like a teaser list you don't need the comments but then as soon as you navigate to the page itself those comments need to be there so a server can't push in a restful world that content along without some extra cooperation same goes for pages as you get secondary pages and then comments for other items you don't know what you need so you still end up in that waterfall system where you're making this waterfall of requests so basically the way that we can solve that is we can communicate exactly what we need if I'm a client I may need know that I need all the posts but I don't know how many to get if I need the schema for a post I don't know if it's up to date if I'm a server I know that there may be 150 posts but I don't know how many the client wants maybe they're just doing a listing of the top 5 basically what I was well what I was hoping to show was that we can communicate using this idea of hypermedia hyperlinks in text and we can send a header that says when you respond to me we look at the bottom of the 200 okay if I go to slash api and I get a response and it comes back with links like api content that's a listing of content I can specify a path in that content that says I want you to push something to me please so if we look at that top one it says links content href we can follow that in and then the server knows that api slash content ought to be pushed right and then you'll see that second push please inside the JSON it says in that thing that you're going to push to me also push to me the next page and also push to me the data comments that are related to it and link those as well and then you can say like I want 100 of them no matter how many items are there per page give me up to 100 of them and I'll be pushing those things so I'll just quickly do a very quick demo press escape so this idea that you have this waterfall here is we click HTTP one and you're kind of noticing that waterfall there are five items per page there and they come in one by one by one and if we were to look at the HTTP two example hopefully they all load relatively much faster once they get there I'm sorry I really messed up the time I thought I had a much longer amount of time do you guys want to see the waterfall or should I just wrap it up okay yeah my slides are available thank you thanks for the hand my slides are available on the website I'll put all these up and also there's a github repo that shows a procty that you can put in front of any jcd api server and it'll start doing this pushes for you I wrote an HTTP client to do this kind of stuff and all the code for the demo is up on github as well and I'll link all that thanks guys I'm sorry