 Okay, we're going to get started here. So this is the rest and serialization in Drupal 8 presentation, and I have to say that there's a lot of code on the slides, it turns out. We figured it was at this point. So that does have the side effect of making it somewhat difficult to read them from further back. If you are interested in those details, you might want to move closer. We also have the slides available on GitHub pages. So if you go to the presentation, the session page on Drupal.org, at the bottom you'll see the link to the slides. Unfortunately, we didn't optimize the images and everything, so it might take a while for them to get down to your computer, but you can look at them there when they come in. I am Lynn Clark. I am an independent developer. I've been working on serialization issues in Drupal for the past six to nine months. I've been working on Drupal 8, and before that I was working on other ways of getting data in and out of Drupal in Drupal 7. And this is Klaus. Hello, I'm Klaus Buver. I'm working for Epico as our company based in Vienna. We're doing the recruiter distribution, and I've been involved with web services for some time now. You might also know me from the approach applications queue, so I do a lot of code review, and I'm helping onboarding new contributors. I'm also a member of the security team. So, first I'm going to start with this. What is REST? People have been really, really excited about the fact that we have REST in Drupal 8, but I'm not sure that everybody's on the same page as to what REST actually is. So I want to do a little bit of just introductory, get everybody on the same page, and then Klaus is going to go into how it actually works as it is right now in Drupal 8, and then I'm going to come back and talk a little bit about the serialization stuff and also talk about what the next steps for making it a more RESTful system are. So I'm first going to start with some pretty dry definitions. For, you know, it's RESTful web services. So Wikipedia defines a web service as a method of communication between two electronic devices over the worldwide web. And so that's really a boring way of saying what you do every single day when you go on the web. You know, say that you have an iPhone app, and that's connected to OpenStreetMap. And, you know, you send a request maybe to get Portland Map data or add a new road because, you know, OpenStreetMap is rewrite. So basically the two electronic devices here are the client on your phone and the server that has all the data. So a web service is basically a way of telling, you know, getting the command or the operation from the client to the server and having communication between those two. And REST is a particular architecture for doing this kind of thing. A style of software architecture for distributed systems such as the World Wide Web. So the guy who came up with this concept of REST, he was heavily involved in ACTP, the development of the early, early web standards. And so he actually came up with REST as a formalization of what they had been doing, a way of describing how the things actually work. So whenever you're using the World Wide Web, you're using a very basic, you know, the minimal set of REST architecture. So I just want to get everybody a picture of what REST in Drupal is right now. So I'm going to run the most basic REST request. I'm going to get a node and I'm going to get it in JSON. So to do this, I go to a particular URI that's worked before. So I'm going to go to a URI and I'm going to use get to actually get the node. And you'll see here that I actually have an accept header. I'm telling it what version of, you know, what syntax I want this node to come back in. I'm going to say application JSON. And then when I send, you see it actually gives me that node in JSON. So there are four components to this REST request. There's the base format, the HTTP verbs, the URI, and the media type, also known as the mind type. And I'm going to go through and explain how these all relate. So base format, that is just, you know, you're pretty familiar with base formats. They're the syntax that you put data into. So base on is one example of base format. And XML is another version. And there are URIs. When you're working with a REST service, you're working with a set of URIs. So URIs can be URIs for a resource. For example, we have entity slash node slash one as the URI that you interact with to work on a resource in Drupal. We're actually hoping to change it so that you don't actually have to have entity in the path. That's something that we would like to have happen. And we'll see if that gets in soon. And you can also have URIs for what are called link relations. So the resource, you know, that's what you're interacting with. The link relation is somewhat different. I'm actually going to talk more about link relations later in the presentation. So I'll just leave it at that for now. So an example of what using URIs and not using URIs looks like is as follows. If you have worked with endpoints before, where you just actually send something in JSON or XML that gives all of the information about what the identifier for the thing is that you want to interact with and what the operation you want to have, what you want to do on that thing is, that is not RESTful. Instead, you actually want to have a URI that represents the thing that you're interacting with. You don't want to tunnel everything through a single endpoint. So media type. People often get the media type confused with the base format. So they think JSON is a media type, which it is application JSON is a media type. But you can also have more specific media types. For example, one that we're using in Drupal in the HAL module is application HAL plus JSON. And what that media type does is it gives a little bit more information about what the consumer can expect to find in the response. So in HAL, a consumer can expect to find links or embedded. Those are reserved keywords and the consumer knows what to do with the data that's inside of those keywords. So you can also, besides how, you know, HAL is actually a standardized media type. It's an IETF draft. But you can also have vendor-specific media types. So GitHub has their own media type, and they have their own rules for what consumers can expect when they receive something that's application-vented GitHub plus JSON. They say that all resources may have one or more underscore URL properties linking to other resources. So that's basically providing the same functionality as what HAL is doing, but in a way that's very specific to GitHub. And so an example that this is something that a lot of pragmatic rest people will actually say you should do, where you actually have the file format. So you have .json, and that's the way you switch between the HTML and the JSON version. But it gets a little bit tough when you have media types like HAL, because, you know, you're basically creating your own file extensions at that point, and that gets a little confusing. So instead, the best thing, in my opinion, to do is to use the exact header and actually spec fights explicitly which media type you are expecting and in the case of post which you are setting. So HTTP methods, these are how you specify which operation you want to have done to the thing on the server. Get means go fetch this page or this resource and bring it back to me. Don't change it. A lot of people implement get in a way where they actually, you can change things using get, but that's not a very good thing to do. Get should not change things on the server. Post, in our application, whereas basically saying, here's some data, go make a thing for me, post doesn't have strict rules, so you can't actually use it in other ways, but a lot of people use it basically for the create operation. We are using patch instead of puts, and patch is basically a way of saying, here are some changed fields, go update them on the existing thing. So this is your update functionality. And then there's delete, which basically means get rid of that thing. And so an example of doing this wrong would be to actually put, or not restfully, would be to put the operation directly in your URI. So instead of that, you actually use the HTTP verb, the HTTP methods to say what you want to have done. So let's look back at that request that I did and see how these all come into play. So we have our base format, which is JSON, the URI, which is entity node 1, the ACTP method, which is get, and the media type, which is application slash JSON. Now if I change a few of these things, for example, I change the URI to entity slash node, the method to post, and the media type, let's take a look at what that is like. You might have to drag through here again. So here's the URI. I'm changing the HTTP method. And here you see I'm using content type, instead of accept for my header, I'm adding application how plus JSON. And now when I send it, you'll see that I actually created a node on my site. The 201 created, that's a status message, a status code, and that tells me that I was successful. I could have gotten other things, like a 5.3d error, or other kinds of status messages, but since I got a 201, I know it's been created. So. And I should also mention that this, what I'm showing you here, this is an HTTP client. It's one for Chrome, it's an extension for Chrome, called dev HTTP client, but you can actually, you can get different browser extensions, you can also use curl, and you can also use HTTP directly from the code in your modules. So now I want to take a look at how this all fits into restfulness. There's a really good model for talking about how restful something is. It's called the Richardson maturity model. And this is a guy named Martin Fowler who you may have heard of before. He actually wrote about it on his blog. So this images from his blog. But level zero, that's basically when you are not doing anything restfully at all. It's called the swamp of pox. It's basically when you're sending XML messages that contain the operation, the identifier, everything that you need, and you're sending that to an endpoint. You're tunneling that through an endpoint. So that is level zero. Level one is actually when you start using those resources, when you start using those URIs to interact with things. So instead of sending the identifier as part of the body, you actually have a URI, like things, which corresponds to your third thing. And we actually, we have that in place. Like I said, we want to change the URI a little bit, but that's pretty much in place. Level two is the HTTP verbs. So that's get, patch, post. We have that pretty much in place. There are a few bugs that we need to resolve, but that's pretty much in place. Level three, that's where we start getting to the things that we haven't fully designed yet. And I'm actually going to get to those later, so I'll just leave it that. The hypermedia controls I'll explain later, but right now, Klaus is going to show you everything that we do have in core right now and how to use it. Before I get into details, we are taking questions via Twitter. There's the hashtag rest. I think it was not a good idea because other people are tweeting weird stuff about rest, but never mind, use it anyway if you have any questions. Yeah, combine it with Drupal.com, that might help. So I'm going to talk a little bit about how rest is now implemented in Drupal and how we use those standards. So what we did in Drupal 8 is a module that is a server. So we're not talking about client stuff doing from Drupal, talking to other sites, but Drupal itself is serving stuff out and accepting that. And to put that into perspective, we have to think about what we are actually trying to accomplish here. And we want to get stuff programmatically into and out of Drupal because humans just browse a page with our browser and we are able to detect what the title is on that page, but computers are done. They don't know where to look for the title of the note, for example. You have to tell them how the data is structured and so we need formats and we need to specify how data is represented in order to make it machine-readable. That's the machine-readable format, like JSON, for example, or XML or other stuff. So as I said, Drupal acts as a web service interface provider. There are a couple of use cases that we can cover with that. I put up a phone here because it's the very obvious reason everybody is talking about rest. You have your native applications and they want to talk to your backend, which can be a Drupal installation. So your phone app is sending those requests and getting data and posting data back. But of course there might also be on-site JavaScript that wants to interact with your backend. That use case, driven to the extreme, is just having a single page and then everything changes dynamically on the page and sending requests back to the backend. You could do that. That's also supposed to be done with the rest module. Or you can use it for migrating, deploying stuff, getting stuff into Drupal. So for example, you set up your staging content on your staging site because you don't want to mess around with your live site. You want to perfectly prepare that node, for example, on the staging site. And of course you don't want to repeat those steps on the production site again. Do it on the staging site, push it over to the production site. That's one use case. And of course there's a lot of people who want to build web APIs so that clients can access their data. And the rest stuff in Drupal 8 is supposed to be used as a private or as a public API. For example, to migrate data from your legacy system into Drupal because it might not have access and can use the migrate module. So the rest module is an application for that. To get a little bit of historical content, what do we have in Drupal 7 right now? The most famous example is the services module. You can do a lot with that. It has... It has a wall from an RPC server, actually. And it was also supposed to be used as a SOAP server. So it's not really in conformance with REST, but yeah, people use it a lot. And there's the concept of endpoints in services, which means you can specify what you or I... Your resources are going to appear, which might conflict with actually hypermedia controls when we're going to talk about that later. And it also uses... It is hard-coded to nodes and commands in users, so there's no support for arbitrary entities. And it does a bit of nasty stuff, submitting forms programmatically because the API in Drupal 6 and Drupal 7 wasn't ready yet to be used generically for a service provider. Yeah, and it has explicit authentication books. So when a request comes in, other models can locate it with an authentication mechanism and that stuff. Then what I did was the REST WS module in Drupal 7 because I wanted to have something different. And it uses the entity API, which is a very cool module. It gives you all sort of metadata about any entity type in your system. So with REST WS, you can expose basically any entity that you have at hand in Drupal 7 to operations on it, for example, on nodes. Yeah, it's not perfect at modern Drupal 7 because we have a couple of problems with page caching because the page cache only works on a URI and when you have RESTfulness, you also want to work on other HTTP headers, for example, the accept header or the content type header. So we have a couple of problems with that. And there's even a hybrid module that uses some approaches from REST WS, the services entity API, which is very interesting. The full power of the entity API working with any entity to the services module I think it's a very interesting approach. And now looking at Drupal 8, what we have there are basically three modules, the REST WS module, the serialization module, and the HAL module. The first one handles the requests and the responses. The second one, the serialization module, transforms those objects, nodes, for example, that are in Drupal and it looks at the database that we load down to just a string, a very long string probably, and that is what is shipped with the HTTP response that is delivered. And the HAL module provides a special format, so we settled on that for Drupal 8. That's our prime format, our canonical format. It has some nice hypermedia controls. It is based on JSON. Yeah, that module does that. I'm sorry. We have a text application language, it's the actual name. So we have a couple of resources now in Drupal 8 and we have operations and we align them in that way, I showed here on the table, and of course a resource is an object of interest that perfectly fits our entity model because entities are also objects of interest, so we expose them as resources. That is the logical stack that we do. And what we here define as operation is an action that we do or that we use to manipulate the resource. In the simplest case, I just want to read something. Okay, I can use a get request to that specific load and then I get a representation of that. So we have the full coverage of all track operations, create, read, update, and delete, and we map them to their corresponding HTTP methods, both get passion, delete, and we also map them to a specific path. As Lin already said, there's still the entity prefix in the URIs, as you can see here. Maybe we can remove that still in Drupal 8, but for now we did it at that path to not conflict with any HTML representation that live at slash node slash one. So we're currently working on URI patterns for entities so that we can be more flexible there. So what we do in the rest of the cleverest module is to expose entities as resources as a set and we use the plugin system for that. And to expose them we use the U entity API. So we basically just have one plugin that serves all entity types that there are, all entity types in core, and possibly all entity types that are added by Drupal contract models. For example, commerce product types or private message entities, whatever. And we have metadata about that. We know which fields are on an entity. So when a new entity is posted, I can inspect here. In that field, I can say that we also made it configurable so you can decide which entity on your side you want to expose and which entities you want to leave alone because you might not want to expose private messages because they're an internal thing. You have a public-facing API just unable to conduct. And we also have access control on the entity level and on the field level, of course. So when you're trying to request a node and you're not supposed to see a field, it is automatically hidden. That is all driven by the usual entity and field access control mechanism we have in core. And then now you might ask what about authentication and authorization? Who is allowed to do what? Who is allowed to read something? Who is allowed to manipulate something? And authentication itself is not handled by REST module itself. Unfortunately, we don't have any additional authentication mechanisms in core right now. Only the usual session cookie authentication, which is a bit clumsy to use from an API, but it's possible. We'll show you that later. And access control is fully driven by the user object of Drupal, so standard permissions can apply. And the whole entity access system is done around that, so we are using that here as well. And there's an extra user permission to access an entity over the API, so there is an additional safety net before you expose just anything about your entities and fields. So actually now it's a little bit of demonstration. It's not a real demonstration, but I put up some code and calls how you can make use of the API. And the first thing you do, of course, is you need to have a Drupal 8 installation. I put up some drash commands here because I use them a lot. It's very handy. So that would be the drash command to install your Drupal 8 site so you can make use of it. Of course, the serialization module and the rest of the services module. I think REST has a dependency to serialization, so you will enable that anyway. And, yeah, we want to work with health, so we also enable that. Just standard stuff, nothing fancy here. Then we... Let's say we want to work with the node resource. We want to read nodes now. How do we configure that? In Drupal 8, we use YAML file as a REST.settings.yaml file. And we copy that from the active config directory to the staging config directory. So that's part of the CMI initiative where you can change your configuration and import it back in. We'll say I've done that. We just copy it over. So there's this long configuration path in your files directory. You will find the file there. Copy it over, then you edit it. And now we want to specify what we want to expose. So that is YAML. You might have seen it before. It's a very simple format. It works a lot with indentation. So the first line resources here defines the array of resources that is enabled. In this case, we just enable it for nodes. So entity colon node is the identifier of the node plugin that exposes that resource. And we only want to allow get request, so we right now get there. This is a very simple format, which is hell in this case. That's basically the structure. You can do it for the entity types. You can enable other request methods and other formats. Then we import the configuration because we added the indentation directory. Now we want to make it active. You can do it with rushEvent. I was very surprised that it actually works with rush already. But you can also do it in a Drupal API, of course. And then we need to clear the caches so that the writes are rebuilt. You can also do that with rush. And it also works with plate. And we need to configure permissions who is able to read that node now that we configure. And you just have to... What we do here is add a permission to the anonymous user roles because we want anonymous users to read the nodes now. And there is a permission for that. We can also do that with rush, but you can do it also in the UI. And for testing, we need to create an example node that we want to get. So I've played around a bit with the devil module, and it has this nice feature of generating content with rush. So that also works pretty awesome. But of course, you can also do it in the UI just creating a basic page there. And, yeah, now we can fetch that. So what approach would be just fetching that from the command line because there are tools on the Unix-like system, or wget. Sending a request, the important thing here is to specify the exact header because you don't want to get actually an HTML. You want to get the hell format. So you specify that. And the UI is important. The actual thing that you want to get in this case, it's just node 1. And that's basically it. But you can, of course, also do it from PHP. So you are developing some PHP application. You can do that with very primitive curl, as you see here. And then you get the curl constants to the UI and the exact header. And you get the chasing back, decode that, and that's basically it. And what's even nicer is to use Gussel. It's an HTTP client. It's a library. It's also part of Drupal Core now. So you just create the client there. It's a very nice object-oriented API. Put in the UI and the exact header as well. You do that and beforehand, and in the end you have this actually made, not before. And then you decode that from JSON and you have your node array again. And that looks like this. So you have a link section in hell. I think Lin is getting back to that later. That requests what it is and where it can be found. That thing, node 1 in our case. And then it lists all the properties of that node. For example, that node ID. All fields, the body field here, for example, with the body value and also with the format. And that's all you have to do in Drupal 8 to get nodes out of Drupal. Get nodes out of Drupal. And because you not only want to interact with single resources, you also want to retrieve whole collections of nodes. And you can do that in Drupal 8 now with views. Because it's a logical step to do. We already have a query builder in core. We don't want to do any fancy query stuff ourselves. So we just leverage views for that. And you have a special display plugin that you can configure that will then expose the rows that views retrievers as JSON or L or XML, whatever you want. It's very similar or comparable to the views data source model in Drupal 7. So if you have used that, it might be very similar familiar for you. And you can either output the whole entity, the whole node, which has collection of full nodes, which could get a little bit big, so you can also just output individual views. That's also possible. So that's very few support. But of course you also want to write to Drupal. Just reading stuff might be enough for some API that want to expose something. Now let's look into how you can actually write a new node to Drupal. I'm doing this here with post request, but it's very similar for patch request if you want to update something. For that, in order to be able to do that we have to go through the same steps as before, copy the setting style to the station directory because we can prepare it there, and nothing goes wrong in the meantime, so don't edit stuff in your active configuration directory. Always copy to the station directory and import it from there because it could break the site otherwise. We just add the post operation there and also the format in which we want to post stuff. And the next thing we have to do is also configure the permissions who is allowed to post nodes. We might not want to give that away to the anonymous user role, so we get to some trusted role here. And yeah, then repeat the steps from before, input the configuration clearly cash, and it should be ready to go. Now it gets a bit complicated because we have to do it with cookie authentication but Gus will actually make it pretty nice, so you just add an array cookie jar here that will hold your cookie and you add that as subscriber to your client, and then you just send the post request to the user login form, and the response of that request will carry a cookie and your client will catch that for you, and the request that you send next will carry that cookie along so you will be authenticated. Pretty straightforward with Gosling Blade. Just specify your name, your password, and the form you want to post it, which is the user login form, and it should be good to go. Yeah, the next thing you have to do is because we are posting things and we're doing it with session cookies, we have to do some CSRF protection because otherwise there are flash plugins and other weaknesses in browsers that could be exploited with that, so we have to get an extra session token. Our client is equipped with a session cookie, so we just request a special page to get the token, and then we just fill out our node, what I'm doing here is just setting the title field and I have to configure the type that I want to create. In this case it's the basic page and content type that we have in the standard installation profile of Google 8. Change the code to OK, and then perform a post request. I also have to specify content because Google needs to know what format is being used, and I have to specify the CSRF token. Yeah, and then I'm good to go. I'm sending in the request, and I will get back a 201 if all went well, and Google accepted the node, or I will get some other status code from HTTP. For example, 4.3 would be access denied, or I don't know if I did probably a 400 if I did the format wrong and a 500 if Google did something wrong. Yeah, you get it, it's just the basic HTTP status. How does that all play together in the REST module itself? So let's take a look inside how that works out. So we have to use those plugins. We use the plugin system because we want to make those components that are responsible for loading stuff, exploring stuff swappable. We want to make them configurable. We want to make it easy so that people can add additional resource plugins in Google 8 country. And the most important thing that those plugins do is separate serialization on the one side and data handling on the other side. And the plugins only handle the data, only handle actual storing and loading data. They're not at all concerned about serialization. So that is completely separate, and that allows you to quickly configure new formats or to format for your particular resource that you want to expose. We confuse by plugins. There are only classes, basically. They're just a class with some methods on them, and that's all that plugin is in Google 8. And what we do is on that resource plugin you have methods that are named exactly as the HTTP method. So for example, if they have a GET method there, it will handle your GET request. If they have a POST method there, it will handle your PADGE method for PADGE request and so on. And yeah, what we're doing for is we have one entity plugin to move them all and exposes derivatives for each entity type. So yeah, that's a feature of the plugin system which is very nice. And as I said, contributed models can look into their and all the stuff or provide new stuff. So there's also a short example in Google 8 core which just exposes watchbox log entries to the resource. And if I perform a GET request to slash dblog slash one I need to configure that before and same as I did for nodes and I need to assign the permissions to access that resource so that's the very same. And that is what's the plugin look like. So as you can see we use annotations in Google 8 and the add plugin tells the plugin system hey, this is an actual plugin and that plugin has an ID and extends a resource based class which is just a helper to get the routes and the permissions and other stuff. You don't have to extend from that you can also write that on your own and overwrite the routes for example. And so the ID gets passed in what we're doing here we're just loading the watchbox entry that ID refers to from the database and then we turn it. And if we look closer you can see when we perform the GET request it translates to the method name as I said the plugin ID translates into the route so the URI path where this is located it doesn't have to be like that it's just a default case where we just take the plugin ID and stick it up but you can change that and here at the end is the resource that we are referring to so that gets passed in so that we know which data to load and which and that's what the response looks like in hell for example well hell might not be appropriate here because it's just basically chasing because that example just returns an array so it just translates it's too very dumb chasing if you want to do more fancy things you might want to use actually class object and then write your special serializers for that so that you can customize this format and make it look more appealing out of the box you also get that get that as XML so what you did you just wrote the resource plugin and you get chasing support and XML support for free which is really nice I guess that's it yeah I have another picture here how that pipeline works now for as module so we have the incoming request that is dispatched by the routing system to the request handler which is part of the module and serializer is an external component and what it does is just if there is a request on the request oh yeah I know I have to deserialize that so I put it in the pipeline and whatever comes out there I pass on to the plugin and whatever the plugin returns I set it through serializer and then that gets back as response to the client and the resource plugin is the only the only responsibility is to talk to the database or do the business logic whatever they have to do for serialization so Klaus was showing you all this JSON and XML and I want to show you how you actually get your data into JSON and XML so we currently have three formats in core we have JSON we have XML kind of sorta it's basically just JSON with angle brackets we don't actually do namespacing or attributes so if you actually want real XML I would suggest not using this one but I will explain a little bit about how you can write your own serialization normalizers and encoders really easily and then there's the hyperattacks application language which is our hypermedia format we went with one that supports doing hypermedia in JSON but it also has an XML variant if you want to use that you can so we depended on the symphony serializer component I actually really like the way that they approached this basically what you do if you want to use serializer you don't have to use rex to use serializer serializer is completely independent if you want to use serializer all you have to do is get it from the container and then call serialize and pass in your object in the format and optionally you can pass in some stuff in context but what it does is it takes that object for example let's say it's a node it passes it through a series of what are called normalizers to take that node object structure and translate it into an array structure and that array structure can be different for different formats and then the encode step takes it from the array structure into the string so in JSON it has the curly braces and all that kind of stuff or an XML it would add the angle brackets and tags so the way this works it gives you fine grained control over how your resource is actually serialized so let's say that we have a node here and it has a text field and a text long field and an image field and a taxonomy term reference basically I'm describing an article here and in your JSON you have field image and let's say that it's actually giving you the file ID that you have as the data for your field image you might actually want to have the URI of the image file itself you can write what's called a normalizer just to change that and the rest of the pipeline is exactly the same just one or two lines of code inside of a class and so the way that this works is there's what's called a chain of responsibility for your normalizers so we have the entity reference item normalizer the field item normalizer the field normalizer and the entity normalizer these are the normalizers that Hal module provides and you'll notice that they're going from more specific to less specific from more granular to larger range and what happens in serializer is it actually goes through some data it goes through and it asks do you support normalization for this object and in this format and within your normalizer you can say yes I support entity reference item field or no I do not support entity reference item field and so in this case because we're working with an image field it would say no I don't support that then we get to the field item normalizer and image fields the image item actually extends field item base so it would actually be supported by the field item normalizer so let's say that you do want to override how the image field is output you just add a normalizer at the top of the chain and then when it calls supports normalization that normalizer will be chosen as the one to use so that's the nice thing it gives you that really fine green control that you're having to change too much override too much all you have to do is add your normalizer same thing for encoders encoders are a little bit simpler but that is the chain of responsibility that gives us such fine green control so I was thinking about talking more about serialization but you know I'm thinking it might actually be better if anyone here wants to write a serialization just come talk to me I'm happy to walk you through the rest of that pretty much that gets you pretty far but if you actually want to get hands on I'm happy to walk through so what's next as I said there's this maturity model and Roy Fielding has made pretty clear that Roy Fielding is the guy who came up with the idea for us he's made pretty clear that unless you have level 3 you have to have level 4 and level 5 most people who talk about rest apis and I think even within the Drupal community some of the expectation has just been level 1 and level 2 but if we really want to be restful we also have to have level 3 so I'm going to explain a little bit about what level 3 gives you and this is another example from Martin Fowler's blog he has walked through how you can actually build iPhone applications to interact with it and the nice thing about that is then the hospital doesn't have to manage all of the iPhone apps by themselves by opening up that ecosystem you provide the way a path for innovation so when somebody were to click this upcoming appointment button on their phone it would get a list of open slots and a get request against the URI for slots and tell it what content type it wants to receive back so the server would return a list of open appointments and if you look here we have one of these URIs that I was talking about before this is a link relation the app would have to know that it's looking for this link relation it's looking for the link relation that allows you to book an appointment and then it gives you the address the actual place that you go to to book this appointment so that's what we're talking about with link relations is basically pointers to where you can go next to do something else and so the client would choose the appointments that they want and they would post a message to that URI and they add their own link to tie their content so basically this is another link relation the client would have to know about this link relation that they need to provide a patient and then they could provide the URI of the patient and then the server returns confirmation it also returns links that tell the client what it can do next so for example we have a cancel here and then you would have a cancel appointment button to go to what URI to go to to actually do that so it is the case that you actually do need to know what these link relations are in order to do something meaningful in your application but there are two benefits of hypermedia one, and this is Martin Fowler actually is the one who I'm quoting here one, it allows the server to change its URI scheme without breaking clients also known as decoupling and it helps client developers explore the protocol also known as discoverability so let's say let's look at what decoupling does for you so an example of a hard-coded way to do to construct that cancel URI is just you know you look at the URI structure as a human process what the URI structure is and then you write in a rule for how it's constructed the problem with this is if they ever change their URI structure which is you know reasonably likely your application breaks so if you use the link relation to access it then you don't have to worry about whether or not they change the URI structure you're not depending on their URI structure you're just accessing the URI itself there's also something called templating that you can do in rest but you can also basically have the rules for creating the URI as part of your message and then discoverability so you know we had this gupation info button as cancel appointment button and those came from the link relations that I know about in my application when I was coding it I knew what these were now let's say you're six months down the line you're looking at a response that's coming in and you see a new link relation like ads don't go what's recommended to do is to document your link relations at the actual URI this is one good thing about using URI for link relations as a developer when you see that new URI just go to it, find the documentation and figure out what it's supposed to do and then you can add a new button to your user interface so that's what discoverability is it gives developers a way to discover and control this is kind of an open question we do have link relations for entity reference but what else could we expose with hyper media controls what else do we want to expose with hyper media controls do we want to do entity management links do we want to do pagination with reviews that is what we are going to be looking at next and if folks have ideas for that we'll be around all weekend and we'll be happy to talk about it very much so there were a few questions on Twitter and we will try to answer those either as direct tweets back to the people or if we can't hear we'll answer them here go ahead Steve we talked about the chain of responsibilities for deserializing data could you talk about how that gets handled for serializing potentially a post or a quote for example of posting patient-slash-lyn-clark watch disappointment I imagine the server would then need to take patient-lyn-clark and get the entity ID of that entity and save the actual entity so does the reverse process look like the chain of responsibility for the serialization process funny you should ask it actually does we that also uses a chain of responsibility and interface stuff it gets a little complex so I will unless anybody is dying to hear I might pass up with you afterwards to explain the details of that one more question are there major items that look like they won't make it into group or buy code freeze we've been hearing this week more and more about the idea of either incremental releases to Drupal 8 that add features or a 9x that Gery's mentioned in his Q&A that could add features but not break any backwards compatibility with what we have in 8.0 are there things in the rest world that could be added to core either in an 8.5 or in this arena well one thing is actually documenting the link relations we don't actually provide documentation at those URLs right now we haven't focused on that because that is not an API change kind of a thing so that's something that could come in 8.1 or 8.2 Klaus do you have any yeah so the biggest pain points is currently authentication in Drupal 8 for the rest of the world because we only have session authentication which is pretty bad but yeah we hope to make it so that you can easily add that in country and also page caching is really painful in Drupal 8 because it's still very broken the same as in Drupal 7 if you have different representations of a resource on one path and page caching is enabled you run into problems it's really present if you want to help with that just talk to me okay thanks hi as you mentioned we can use the accept headers to ask for a specific format of a resource so how far has content negotiation been developed in Drupal 8.0 so do I have to ask for a very specific media type or can I do things like okay give me XML if you have it if not give me JSON is that already there or is that still in development that as far as I am aware is still in development we basically if you send a request for application JSON and you have other preferences I think it sends you a 406 if it doesn't have the one that you specified first I think that's yeah we have no fancy content negotiation in place so if you send multiple stuff I have no idea what happens probably the first one gets chosen I don't know so just please send one thank you guys one follow up question with respect to page caching I mean is it only broken if I use the building cache or can I use vans and say something like very accept yes you can do that but the core needs to be aware of that as well and currently core isn't aware of it so it's a problem show question for the link relations the examples we showed they actually had math.org in there does it really depend on the base URL in Drupal or is it something I can define because I'm worried about station developments right so it is currently automatically created based on what where the site we could look into figuring out how to change that if you want okay well I'm just thinking because then I would have to change my client not only the URL actually the whole code to search for the link URLs if I worked with the desktop mm-hmm yeah you would but let's fix it I mean we can do it I answered this right to the first question but I just wanted to hear more one of the things I really like about the services model the authentication plugin ability because we have clients that have to authenticate with basic off, SAML and cookie off and we actually accept all of those and it's really nice to be able to just define your own authentication plugins and so maybe you can talk more about how you're going to handle that with Drupal 8 because cookie off only is going to be a major blogger in adopting this yes it is so we are trying to keep it out of REST because it itself might not be concerned about that and other system might use the same mechanism as well so what we will try to do is to make it really a core feature that can be used by possibly other modules and yeah there will be some mechanism to look into the system and check the request for other authentication credentials but yeah we also need to make the page cache aware of what the authentication plugins are looking for because the page cache needs to vary based on but this is an authenticated request or not but yeah that's the biggest problem it's currently it's a little bit broken but we're going to work on it if you want to help just step up I'm just wondering if we can make output requests to IEU ID for content deployment so that actually ties in with another question that was on twitter which is why we are not supporting foot and there's actually a couple of good discussions as to why we're not supporting foot one is something that Larry posted called putting off foot I'm going to tweet the URLs for these so he posted an explanation of why we are not supporting foot and there's also Roy Fielding in 2009 posted something called if okay to use patch where he basically said you know I actually want to use foot so I just wanted to mention those using foot for content deployment so that you can actually have the same identifier or okay so you can have the same local identifier because like UU ID you can actually deploy that without using foot but if you're looking for the same local identifier then you might need to use foot for that if it's to be restful then you need to be in the URL instead of just I think we haven't decided to use UU ID in the URLs because it's so long and not really human readable and a bit of a pain and also the problem with UU ID is that they might not be unique overall entity types so they try to be but there could be collisions I mean the probability is pretty low still but it makes the system also more complex which entity type is actually going to be posted for example or going to be updated so you would have to iterate over all entity tables to find out what is the thing okay and I just had one other question are we hoping to one day serve web pages through the west the restful module so you're talking about like having blocks in addition to the actual node I mean the HTML would serve directly to the web browser should that be served via the restful module yeah yeah so the comment was there should be no distinction between the API and the usual HTML page and I agree to a certain extent but the problem is we serve HTML mostly as pages and that's more than just a node there are blocks around it, there are menus and whatnot so that is actually something different I guess and yeah okay thanks very much thanks I just wanted to add one of the complaints that people have around services or Drupal score age acts and stuff like that is that Drupal has like fully bootstrapped just to like spin the stuff out of the theme system and all that kind of stuff is there any developments on your end or whiskey in general that's helping with that problem yeah of course the whole whiskey project tries to split up Drupal core and especially Drupal bootstrap that it is more lightweight but yeah we are operating on the fully bootstrap Drupal the current implementation in Drupal 8 core okay so yeah we have to load stuff that we need so we can avoid that the entity API has a lot of dependencies because it needs to know about stuff so yeah thank you the whole point of API as an restful which for your services is for machines to reach your data to consume your data don't you think that by basically providing inconsistent URIs like for nodes, commons, blogs users and so on and so forth it gets hard for other developers to write code that will actually consume that data so you are saying that we should use the same URI structure for both nodes and users so and you wouldn't have like node in the URI you would have entity and then for example URI here and you would be very simplified unified and basically anyone who is writing an app would be able to consume that data we are using the same code instead of writing this you know so yeah what I have seen as best practice is to actually specify what kinds of entities you are working upon I think that is actually considered developer experience when when you actually specify that you are talking to a certain kind of entity and also you wouldn't be able to reuse the code totally because different entities might have different link relations so there is a lot of stuff that actually is specific to the domain model of that particular entity so you wouldn't necessarily want to have something that is generically working across users and you know everything else and it would be very slow because you have to search all entity tables for that URI performance aside I think what we are doing here now is actually making it harder to develop apps and to actually read data that is supposed to be easier to read because like if I am writing for example a piece of code that is supposed to consume 5 entity types the accelerometer with node blocks users has to basically modify my code 5 times and you can walk to the code that will retrieve that data very easily I am going to table that discussion if you want to continue that discussion we can continue it but I think that there are a lot of people that would disagree with the idea that that improves developer experience there is a lot of blog posts that actually recommend the opposite so I think that is a discussion we can have maybe afterwards second question is you are writing staging content is it like in the works or what is the status of it because you do need UUID in order to stage content you can't really stage it with local IDs so we have UUIDs and they actually do basically the UUID is actually ingested and used when you are posting more questions there were a few people that actually asked questions on Twitter I would be happy to go over those questions now if people want so yeah there were people asking about the put impact and I will post those to blog posts but basically when you are doing a put there is kind of a strict semantics to put you are supposed to replace everything and so that is why Roy Fielding actually said a lot of times you want to use posts and not put and so we figured that we don't know what people are going to be doing what field people are going to attach and whether or not they are writeable or not all of these other things so it is safer to go with patch we can actually abide by the semantics of it more more faithfully somebody asked if there was a crud for views basically you are going to be able to edit your views using and that is a config entity and we are not really supporting config entities configuration is entities like views are configuration entities it might be possible I haven't tried it actually but it could be feasible one of the arguments we actually had while we were discussing that was configuration you can already do it in the ammo and then use get or something to get it out to your site so it makes more sense for config to actually manage it in code rather than over HTTP so I just have a quick question the dblog example you showed is that how easy it is to create a custom resource that is maybe not an entity that just exposes it yes at least I hope I guess people will run into problems but yeah we try to make it as easy as we can get so there is another question if I use views in d8 to limit fields can I still load it from the cache and I actually have not looked deeply enough into the views integration to say I don't think so but I think you can just use the standard views caching so views if views retrieves all that rows and caches them somewhere that should work the same for the rest display plugin so just the standard views caching yeah because it's using get items in both cases on the function get items and views so whatever that does we have access to the cache so yes somebody asked you to talk more about our use of csrf tokens and specifically the fact that it seems to be a carryover from form api yeah it does something similar like form api because you have to verify that the post request that is performed is actually made by that client and not on behalf of someone so that they usually exploit to some external site and it executes some javascript or some flash and you're just visiting the site that the javascript is already executing and then sending a post request on your behalf to your actual site and what happens if you are administrator at that site that request will be performed with your administrator permissions so it could do a lot of harm for example delete request, delete stuff or update stuff in a very not nice way on your behalf that's why we have to talk back to the server to retrieve the token and then add that to the request so that it is actually an original request that is intended to be sent a couple of other frameworks do it the same way i guess ruby and jango use a similar header is not the same for post requests that are performed with cookie session authentication so basically we just took it from there but i was wondering is it possible if you want to get node there from the rest of the api but you don't want the full node object if you want to say something really simple like the node id advisable to strip out the other fields there are different apis that do that there are people who talk about how to do that basically using get parameters we are not currently supporting that that's something that could be supported in contrib i don't think that we're core we're trying to limit what we put into core just because we have to maintain that through the years and so for some of this experimentation we're really looking to contribute to some of that experimentation but yes it's definitely something that people do and that you could do the comment was that in rest it's supposed to be a granular api i think we tried to be as generic as possible and just output everything that the user has access to by default and if you want to output something different just rip out some fields for example i guess you just write your your own normalizer the normalizer for the serialization system and then you can easily get rid of that i guess a lot of people will do it because there's a lot of crap on nodes that you might not want to show to your public API consumers consumers another thing that you could do is actually write something generic that is a resource plugin and then prepares the entity before it gets serialized and that would be another option any other questions? there was one person asking about a little authentication so i don't know what the question was i think the question was just if i could elaborate more on gustal authentication i'm not sure if there is a special handler in gustal that deals with authentication later on for that demonstration with session authentication so i'm not really familiar with that the internet will do just that hey, any more questions? we're also happy to answer your questions afterwards but we're here as long as you guys want to give us questions