 So this is exposed Drupal with RESTful. This is the session you walked in. If you were not looking for this session, please stay because this is going to be cool. I get over excited about these things. I have to warn you about that now. And yeah, all this is about Drupal 7. You've been hearing about Drupal 8 all day. This is about Drupal 7, but it's going to be available on Drupal 8. So this is not the title that I wanted for this session. But apparently, the Drupal.org session creation note has some validation. And it doesn't allow you for titles this long. This is what I want to present. A comprehensive guide to build an HTTP API without hating yourself six months after release. And this is because I've been building and designing APIs for quite some time now in the last two years. And I've been having some hate issues with myself. Especially when you design an API for the second time and you say, OK, I'm going to be smart. And I'm not going to be hitting the same problems that I had the first time. But then you hit some other problems. And you don't solve them perfectly. So I decided to start gathering all these experiences and putting them on the RedFool module. And yeah, that's what I've been doing. So this session aims for two simple goals. If you walk out of here, I would like you to have these two concepts in mind, if anything. So first one is that REST is not just exporting an entity. So it's not just taking a node, doing JSON and code, and putting it out there. That's fine. That covers lots of use cases. But REST is not only that. I mean, if you want to create a simple widget for your site, maybe that's enough. But that's not REST. REST is full crud, create, read, update, and delete. It's also authentication. You wouldn't want to serve to your anonymous users the content the same way that you serve them to authenticated users. So you have to deal with authentication, with access, who can view that, who can create what. So there's authentication. And how do you authenticate an HTTP request where passing a cookie feels very awkward? So yeah, there's a lot of other stuff that I'm going to be talking about, which relates to REST, which is filtering listings. Sometimes you don't want everything, or doing pagination, ranges, et cetera. And the other concept is that RESTful can do a lot of that for you, for free. And by free, I mean free in all of the senses. Some of the cool things of the RESTful module are data providers. So when you're exposing stuff with the RESTful module, you are not limited to entities, which is mostly what you're going to be doing the 90% of the time, but you are not limited to that. You can expose maybe a database table, something custom that you had to do because you know how projects work. I don't know why you have to expose a table, but you can do it. A use case for that could be menus. Also, another thing that you could be exposing, and we are doing that, is exposing the plugin system, either if it's C tools or another system. So you're not exposing anything that's actually in the database, but something that are files, and it's there. So you can basically expose whatever you want, and all of the principles that I'm going to be talking about apply to those. Another benefit of the RESTful module is the resource composition and what I call the URL query language. These two things, I was actually very happy to see how GraphQL this morning introduced these topics. A resource composition is basically when you have an entity that relates another entity. It's super annoying that you have, let's say, an article that has three tags. And then when you get the response for that article, you get three tag IDs. And then you have to do three more requests for every tag. And if you stop there, that's cool, but maybe you have to dig further and keep making requests. So making HTTP requests is often expensive in terms of performance. So resource nesting, or resource composition, what it does is it takes the resource and embeds it into the original response. So you would get JSON output with the article and the JSON for the tags in there. And that's pretty cool. So imagine this clone of TripAdvisor. You had this idea, I'm going to clone TripAdvisor and I'm going to become rich. And you call it Voyage Consulier. So I built this slide for Montpellier and it was cool that it was in French. So in Catalan, it's Consulier de Viaggia. So the site is to find restaurants. And restaurants have deals that apply to them and all this stuff. So you decide that you're going to build this with DuPo because DuPo is an awesome content management system. And it allows you to build your content model to do it in a structured way. And it's awesome at dealing with that. So another thing that you decide is that you are going to go decoupled. For there are several reasons to do it. So you analyze your reasons and you say, okay, I'm going to have to build not only a site but also an Android app and an iOS app. So I'm going to build it decoupled. And the first thing that you realize that you need is an HTTP API. So when you're designing an HTTP API, what you're basically writing is a contract between your backend team and your front team. So you as a DuPo developer or a backend developer, you sit at your desk and you think about the data model. In this case, you think about how a restaurant needs to have a title and a description and also needs to have relationship to the deals content type only that you don't think in content type ways because this is decoupled, right? You could take out DuPo and with that API designed, put something else in its place. So you write the API design as a contract. It's basically the promise of the backend team that you're gonna deliver that set of URLs, in this case could be slash restaurants slash ID and that hitting that URL is gonna return with a JSON payload. In this case, it could be ID equals a number and then label equals this is the best restaurant in Barcelona and the description could be, I don't know, the description of the restaurant. So you document that, you deliver this to the front end team and then you start working but the cool part is that the front end team can start working at the same time in parallel building on the assumption that you're gonna fulfill your promise. So when you're building this HTTP API or you're designing it, rather, you should be avoiding all of the DuPolysms that typically get in your way and that is because, as I said before, when you're building the couple sites or the couple apps, you are able to swap out any of the parts even if it's the back end or the front end and replace it with another thing or maybe introduce some duplication for when one is down, you have the other one. So imagine a situation where you don't think that DuPolysms is cool anymore and then you implement that with a Node.js app using Express. It would be very lame to have to implement in Node.js an API that fakes that it's a Drupal installation because in your API design you have things like use a slash node slash node ID because slash node doesn't make sense outside of Drupal, right? What you really want is slash restaurants and then probably a UUID instead of an ID because if you use auto incremental IDs then people can start guessing but that's no stuff. So that's one of them, they're a bunch but the most important one is or I get annoyed by the fact that I don't want to have field underscore and then the name of the field. This morning I was trying to understand why this was happening in field UI but you really want to have the control to say the output of my restaurant JSON document is tags or city or label description, something clean, something that doesn't leak Drupal all over it. So the last one and this is my favorite. You really want to have your description and then have your description like description equals black and not, and I have to look at this, have description be an object with a key of und and that be an array of one value that contains an object that has a key of value that contains black. So these are some of the goals that you need to achieve by doing a good API design. So another thing that I learned through sweat and blood is that you really want to version your API. So if you don't version your API, it means two things. It means that you assume that there will only be a single client ever for your API and I guess that you cannot really know that for sure and it also means that you're a bit masochist and you like very painful deploys. So let me explain that. So imagine this situation where you're building Voyage Concealer and you have your API and you release it to the world and create your app. Let's say that you're building an Android app and it's working but then you get a new feature and you get to evolve the API and you know that evolving the API means that you're breaking the existing code but it's okay, you haven't deployed to that. It's breaking the existing app but since you haven't deployed that nothing happens yet. So you work in your dev servers and you work on the Android app code that uses the new features and everything works, right? So it's time for deploy. You need to deploy both the API and the Android app. So do deploy the API first. At that point when you have deployed the API your existing Android app, let's call it the old Android app is not working because it's not 100% compatible with the new API, right? But it's okay, you can assume some downtime so right after that you go and deploy your Android app, your new one that's gonna click together with the new API and it's gonna be awesome but something happens, the deploy doesn't work as expected so you don't panic, you roll back to an Android app that doesn't work with the new API, you don't panic, you roll back to the old API and then try to figure out how you have to manage all this, the new debugging with the new API and the old API, so that's a big mess. So the simple thing to do is that you create the new version of the API while the old one is still being served and existing, you deploy the code but your old Android app is locked to version 1.3 of the API. So it keeps working, you don't have even downtime when you do the deploy and then you do the release for the new Android app and it's breaking but it's okay, you revert and the old Android app is using version 1.3 even if you have deployed version 1.7 or whatever. So that gives you that flexibility and also having versions allows you to release the API to the world and have people build clients for it and you are happy because they're doing your job for free and then they can lock to different versions depending on the time where they built it and the features that were available. So having your API version is like the basics, you cannot get away without it and of course all of these thoughts are introduced in the RESTful module. So we have been working on the RESTful module for quite some time yet but the motto of the module is that we are always looking for the practical use case. So if I were to propose a feature, a crazy feature, I do that sometimes. And I put it in a pull request, Amitya would ask me, do you have a real use case for this? So we try to keep that in mind but at the same time we try to stick to the best practices. We research what other people are doing, we research what the academic way of implementing this is but without obsessing about getting a medal for it because nobody gives medal to developers anymore. So one of the first things that you will notice if you use the RESTful module is that it's developer re-ended. So when I was writing this slide, it came to my mind a conversation that I had some time ago about how great is Drupal that allows non-technical users create awesome digital experiences and websites without having to code anything with basically programming with a mouse. And that's really impressive and I love the fact that you can download the MetaTax module, enable it and go to the node and just type the MetaTax and forget that those, did those go inside of the body HTML tag or was it in the head? And do I have to put the value inside of value equals, I don't care, I have the MetaTax module and I just do use the UI. So that is awesome. But then there's also the example of the migrate tool module which shifted from having to maintain a very big UI for creating migrations with Table Wizard and all that because all of the complexity in those cases lives in the UI. So they shifted to a way of doing things where they just wanted migrations to be classes and those got run because a migration is a complex thing and you don't want a non-technical user or non-technical person doing your migrations, right? So we think that doing a REST API requires this kind of technical knowledge. There are other solutions in the Drupal world that don't and allow you to create, to expose some fields and some stuff using a mouse. If you can get away with those, just do it. If you don't need any of this, you don't have to worry about this. But if you want to have control and you want to use the cool features that are implemented in here, you know that this is oriented to developers and this slide is very red. So in the RESTful module, we do everything in plugins. In the version 1.x and the 2.x, we use different plugin systems. In 1.x, we use the Seedles plugin and in 2.x, we use a back port to Drupal 7 of the Drupal 8 plugins, which is pretty nice. And it's very easy by everything is a plugin. I mean that a resource is a plugin and authentication provider is a plugin and yeah, everything is a plugin. So if you have to, if you're not satisfied with what it comes out of the box, assume that you can write a plugin that has most of the work done for you, that you can implement only the things that you want to do differently and then you can use it. So creating a plugin is not that complicated. And oh, let me go back. So I added 1.x and 2.x, like these small batches at the bottom of the slides. So you can know that I'm talking about one version or the other when I'm presenting a slide. Basically, I'm gonna be talking about both at the same time, most of the time. So how do I create a plugin? So the version two uses the Drupal 8 style plugins and this is called an annotation. So to create this, you just have to create a class file. It's just a regular class inside of a directory. You will learn how to create that and you will use it for the eight plugins. But the key is that you have to add an annotation which is a special comment. Do you see this add resource in there? That means that it's an annotation. In there, you provide some metadata. So the important parts here are marked in green, orange, and blue. So you can see that the name contains the word articles. That is the resource identifier and that's what's gonna be used by when you are hitting a URL. It's gonna be slash articles. So 1.5 is the version. So realize how hard it is to create a new version. You just create this and instead of 1.5, use 1.6. And there you go, you have your new version and you can still hit the old versions. So apart from that, we said that we had what is called data providers, that we would be using the entities most of the time. So in here, we are using the node entity type with the articles bundle. So that's 50% of the work that you have to do to expose a resource. So it's not that hard. The other 50% is this. Basically I said that you should be able to have control in how you expose things. It's not just a straight serialization of the entity. So you require some mapping. And mapping a field, it's as simple as creating an array and returning it. In this case, what we are doing is we want to have public property called cool text when you're exporting an article. And you want the content of that property called cool text to be extracted from field underscore cool underscore text. So that's it. That's how you define a resource. And when you're done, you get a lot of stuff for free. So when you're using an HTTP API in the real world, you don't only get to hit the resource slash ID, right? You need to do listings. So give me a list of all of the articles that I have. But give the list sorted by a particular property. So if you exposed a property called cool text, you could sort a list by cool text. If you expose the creation date, you can sort by the creation date. So basically when you expose that array, you're not only providing an output, but you're also providing ways to sort, filter, and range, and do a lot of stuff. So sort is pretty straightforward. Filter, you can filter by any property. So if you want to get all of the articles that have a title of this is a cool article, you can do that. You just use in the URL filter equals, filter title equals cool article. And that's it. We'll see an example later. So after that, there is the range, which is basically the front end team. They don't want like a gazillion articles when they are building just this slider or carousel. I don't know the front end lingo. So this thingy where they have a thumbnail and a title, but they only have four of them, they provide a range, right? Or a limit, max number of items. So they say, just give me four articles. So give me 50, because I'm only going to use four. And the cool features are the sparse field sets. So when you're getting the output of a big content type and who doesn't have a big content type with a lot of fields, right? The front end gets a giant payload, gets a lot of data with all of the fields, even if they only need the thumbnail and title. So with the sparse field sets, what you can do is specify which fields you want to have in the payload that the server is returning. So you can limit and tailor the response based on what your needs are as a front end client. So that allows you to do a lot of performance improvements, especially when you combine that with resource nesting that we talked before, like getting the related entities inside of the original entity. So with those combined, you can pretty much get tailored responses with the client needs. So that gets you to a point where you only have to design the API and let the front end design what they want to get, right? And if the design changes, the design of the Android app, you don't care. They just make the call differently and get the data that is available. So that was for reading, but for updating and creating entities, there's also good stuff that you get for free. You get entity validation. And entity validation works with a separate content module, but the idea here is that entity validation in Drupal only works at the form level. So you're validating a form. You're not validating an entity. So that's in Drupal 7. So what you're doing with the entity validator module is you create some rules, saying describing my article to be valid has to comply with these rules that I'm defining. So when you're creating an entity with the RESTful module, what you do is if you have the entity validator module enabled, it validates your entity for you and you get a cool error handling back saying that, you remember the cool text? It wasn't cool enough. So sorry, try harder and improve your entity, right? So that and resource nesting. You don't have to create the three different tags before you can create the article to get the IDs to put in there. You just get the JSON object with the article properties and the property for the tags and everything that's created in one request. So make improvement. So this is just a representation of what I mean by resource nesting. You see that you can have a meal and a particular meal can have or will have an ice cream and ice creams can have different toppings. So imagine if you have to traverse all that, making HTTP requests, that would be lots of them. So this is what you get when you get a list of articles. So I'm hitting a slash API slash articles. So that means that I'm requesting all of the articles with all of the fields for the articles with the default sort and no filters. So I'm seeing there that there are three articles that come back. The first two only have ID label and body and the last one also has some tags. The first two don't have tags because no one tagged them. So this is what you get and with the URL you can control what you get back. So in this other example, what I'm asking for is give me all of the articles that have the label of Lorem and you give them sorted by ID. So if you see there, there are only two articles instead of the three that we had and that's because some of them don't meet the filter of having the label equal Lorem. And you see that the first one is ID five and the second one 42 because they are sorted by ID. And notice here that you always interact with the public fields. So you're not saying sort by node ID. So this is always shielded from the front end developer. So Drupal is not leaking outside. So after that you specify that you only want the fields ID and tags. You don't care about the label because you know it's gonna be Lorem. You're filtering for all of the articles that are Lorem. So yeah, you can get a really specific and build the call that you need for the view that you're building from the front end perspective. The RESTful module is working with versions, as I said before, and there are typically two ways where how people implement version negotiation. And version negotiation is how the client or the consumer of the API tells the server, hey, I want version 1.3 and not version 2.0. So some people do it in the URL. Some people do it in headers. Both of them have use cases. In my opinion, both of them have valid use cases. So in the RESTful module we support both and in an interchangeable way. So you could send a URL to your non-technical PM with the version in there because you don't want to explain him how to pass a header to the browser. So the negotiation. Now you just give them a URL and they hit that and they see the results. So yeah, you can use both. And also, if you want to negotiate the version inside of the media type, you could do that. So one of the features that are imported from Drupal 8 are the authentication providers. This is not in Drupal 7, so we went away and borrowed the concept from authentication providers from Drupal 8 and we implemented that. So what that means is that you have a way to authenticate your requests and you can do it in several different ways because remember, these are plugins. So one plugin will allow you to authenticate your requests using cookie authentication. That means that if you send a session cookie along with your request, you can authenticate yourself as an admin and get more fields or maybe you can get access to create some content type. Another one is basic auth, which basically means that you're sending your username and password along with every request and you should only do that on encrypted connections, by the way. And there are another one that's called token bearer which is passing along a token. So there are different authentication types and you can build your own. So if you have a use case where you only want to authenticate requests if they send the username and password and a picture of a cute kitten, you can do that. Or you can implement LDAP, which would be more useful. Another feature is rate limits. This is basically saying, okay, this role, the anonymous user can only hit my API X times during this period of time. So you can say maybe they can only hit your API 20 times in an hour or 100 times in a week, et cetera, et cetera. This is to avoid DDoS attacks and it will even give you a nice message. Like I think it's 429 error code telling you that you exceeded the quota. Try again and then we'll tell you when you will get the new period for getting the response. So again, this is a plugin so you could implement a plugin that gets, you know, during nighttime, it allows you to make more requests than during daytime, et cetera. I have to admit that I've been lying to you the whole session because I've been talking about JSON all the time. But you don't only get to work with JSON because you have output formats. So what that means is that we have output format plugins that will allow you to expose your Drupal through XML, through JSON API, which is a JSON specification. And yeah, whatever you want to build. You want to build JAML, expose your Drupal with JAML. You can build that. And basically, what that means is that with RESTful you only worry about gathering the information. And then you defer the format to the client. So they can ask, give me the stuff in XML because this is an Android app that uses Java in a framework that works well with XML. That's fine, they decide. But at the same time, the same resource can be served in JSON or in HAL JSON. So yeah, this gives a lot of flexibility. So we only get the info and then with output formats, we can show it in different ways. And this is where your mind should start blowing up. So all this, it's auto-discoverable. So what this means is that you don't need to write a single line of documentation. Don't write documentation, just allow people to discover the API. So when you browse the API at the root homepage at slash API, what you will get is a list of all of the resources that are available for that site. And I realize that you cannot read this, don't worry. This is just a list of all of the resources that are available. And since this is just another resource, it's a resource that serves resources. You can do filtering, you can do sorts, et cetera, et cetera, et cetera. So here you can discover that you have, let's say, an article's resource. But that doesn't help you much, right? Because you want to know what's inside of the article's resource. So by doing an options call or an options request to a single article, you will get an explanation of what is inside of the article. So options is just a regular HTTP verb like get and post, et cetera. So do you get something like this? Don't try to read it. What you get is, okay, you got an article's resource, but what's in there? And doing this, what it gives you is, it gives you the information that there's a label. So this is all the information for a label. And this label, you have this property that is called label, has this description, which could be, this is the label of the resource that it's implied from the title, et cetera, et cetera, some documentation there. And then things get interesting because it gives you information about the schema of the property. Label is a string of 255 characters, which is not required and it's read-only and it's cardinality of one. So doing this allows you to introspect the API and know that label is a string and you should expect this to be a string and you can assume that you can pass this to the front end and they know how to deal with this. Other things that you can do, and this is also out of the box, is imagine that you also document a form element for this particular property. So for a label, you would build a text field of length 255 with a placeholder and just like you do it in the form API. So that allows you to, as a front end developer, build a library that reads this and creates forms automatically. Instead of you going there and writing DHTML in a React template for every property that you have in the articles, you just build a library that reads this so when you add a new property, that library automatically creates the new form element. So that's only an example of the cool things that you can do. With introspection and autodiscovery. So this is out of the box. And remember, all this because you wrote an annotation and a single mapping. So if you liked the GraphQL example this morning, you are also gonna like JSON API because JSON API, it's basically a specification. They say that you can use JSON API as a specification. They say that it paints your bike shed, which basically means that it just does what you otherwise would have to discuss a lot to get to a similar solution. So JSON API has a lot of cool things that I don't have time to explain in four minutes and 25 seconds. So go check it yourself. I put up there some demos. I don't want it to do a live demo because that's scary. So I recorded some videos. I put up some code. The code is very simple. It's just a feature that spices up a little bit the articles and page content types to get it more interesting, to have entity reference fields, et cetera. And then another module that creates the resources for those spiced up content types. And then when you have that installed on your side, go and watch the videos, which are silent, by the way. But you'll see, hopefully you will get the point. So you coming back to the meals example, you can do very cool things like, and it's explaining this is super hard. So it's okay if you don't fully get it. So you could do things like, okay, give me all of the meals, which have ice creams that have a topping that has the name of chocolate, right? And that may seem stupid, but when you go to the real world, you may have to list videos that belong to a TV show that is marked as being available for Apple TV. So from the Apple TV, you want to only get the videos that belong to TV shows that you have the rights, the legal rights to show them in an Apple TV. So doing that with, you know, simple exposing click and bidon systems, that is very hard. And, you know, there are other combinations that you could be doing, sorry. In that example, what you do is you call the slash meals and then you filter by ice cream dot toppings. So you basically traverse the relationships through the name of the field. So meals, half title, sorry, a field of ice cream and ice creams have a title, a field of toppings. So when you want to filter by nested resources, what you do is basically you traverse by the field names. But, you know, you could also want it to, no, no, no, but that's not what I wanted to do. I didn't want all of the meals that had ice creams that had a topping of chocolate. I just wanted all of the meals and then limit the toppings to the ones that are chocolate. So these things get crazy and there are a lot of use cases and if you think that there are some that should be included, feel free and reach out and go contribute to the issue queue and that's it. Also, before I forget and we'll go to questions, there's spins on Friday and don't miss them and if you liked what you saw today, go and do an evaluation on the session. If you didn't like, just forget that I showed you this. So any questions? Hi, firstly I'd like to say that I'm using it and it's amazing and amazing simple and it's suit is in its best. And my question is, could it be possible to create entity bundles using post requests? So you mean like creating the entity bundle, like making it available to Drupal, like you have to create a new content type? Yes. Yeah, I don't see why not. Maybe there are some nasty details that I'm overlooking but you could make a data provider but that would be not out of the box. You could make a data provider that interacted with Drupal's API to create the bundles. So basically what you would end up doing is having a custom hook entity info alter that reads from the records of the resource and then it creates the bundle. It would certainly be scary to build but yeah, I think it would be doable. Another question, you said that you are using C tools plugins generally but I think that some things, there are entities like the authentication, I think. Because you are setting them as entities there, is that true? I'm not sure I got that question. The authentication plugin are entities, not C tools plugin. Oh, okay, so yeah, we're maintaining 1.x and 2.x and the line gets blurry. I think in version one that... In version one, I think that C tools plugin I may be wrong, I would need to check it. Okay, Amitya says that it's a C tools plugin. So yeah, I think those are also C tools plugins. App stickers for everyone that asks the question. Hi, I apologize, it may seem stupid but from what you showed today, I don't see why we need a special module for this. Why aren't we using views for this? Like you create listings, you can sort, filter, select range and you just go from a URL. Why is there a module for this and not views? So if you are not missing anything with views, then you don't need this module. But if you need to create something, maybe you need to allow users in that but yes, consider your example to create a deal for the restaurant or to add a comment. You cannot do that with views, right? If you want to throttle your...