 I can't walk on music. That's very exciting So I think up. Good morning. Thank you all for coming to my talk and thank you for that very kind introduction I submitted a talk to word Campeja in 2019 and getting that talk accepted was one of the most exciting moments for me professionally that I can recall and This has been a long time coming since the talk that I Submitted though. I think has benefited from a little bit more time to bake So now that we've actually had all of these fine organizers put this whole conference on a second time Hopefully this will be a little bit more relevant for the present day Actually, let's get a round of applause for the organizers It's enough work to organize a small local word camp much less the regional one much less doing that one twice so My talk in 2019 will 2020 was going to be called the past and future of the WordPress rest API and it was going to be a Retrospective on the work that we did to add the API into core to write the plug-in and To acknowledge the things that we think went well the things that have stood the test of time as well as the things that have maybe proved to be either mistakes or issues that we have to work around and As I was reflecting on that topic that I came up with previously I realized that I had been saying basically the same talk for a long time I had started when we were advocating that WordPress needed a rest API to be able to take our data and Flow it out into different applications and that has happened We have enabled page builders and we have enabled static site generators. We have enabled marketing integrations But the most important thing that the API has done is of course become the foundation for the block editor And the block editor is the future of WordPress. So I think that I'm actually asked the organizers yesterday to change the track of this talk from rest API to development because Increasingly, there is no development in WordPress that does not at least in some way interact with the rest API It's how we get the data one of the ways we get the data from the database in PHP through to the JavaScript on the client and I think it's very important for us to have Consistent understanding of the best practices for using what's in WordPress core So if I may be so bold as to retitle my talk I'd actually like to talk about getting the most out of WordPress in the sense of actually getting that data out of WordPress as easily as possible So if you are comparing what I'm saying to the description on the schedule You might notice that I have a laundry list of topics there that probably would take four talks to give in full and We're going to have to reduce the scope a little bit to focus on two here today So the goal of this talk is to talk about API performance and to talk about how to get faster responses And then to look at schema, which is one of the key Architectural concepts we have inside the API which I believe plug-in developers might find useful there is a lot more that we are not going to get to cover and I would invite anyone to ask me questions about any of the other topics that we could be discussing today in the hallway track at The party online after this and also I'm honored to be going on after a Documentation team lead because that's the most of the work that we need to do and we need to collaborate Between the Gutenberg plug-in development team and the core Committer team to really improve how we describe these patterns for people in the community to be able to use So with that out of the way Let's talk about performance anyone that stuck around for this talk because they didn't want to see Alan talk about performance next door I'm sorry. It's a very very important topic in the project right now If you've been paying attention to the development of the last couple of releases One of the major topics has been how do we get WordPress to operate as quickly as possible both in terms of Rendering the site that the viewer sees on the front-end, but also how do we make the block editor very quick? How do we make the experience of using WordPress as an editor efficient as well? and Unfortunately Partly because it's easy and we'll explain why The thing that I see the most is people taking WordPress rest API data and Looking for the biggest bucket they can find They want to find something that gives them every possible piece of information at once Everything everywhere all at once is a great movie. It is a terrible data architecture process So let's take a interface that any developer in the room or watching on the live stream. Hello Is going to be very familiar with we're going to replicate the core WordPress loop the list of recent posts in A application that's driven by the rest API The most common way that I see people do this is that they hit the posts endpoint Which is where the post live that part makes sense and they give it this embed parameter, which we'll talk about in a moment It's not a lot of data. It's a hundred and sixty kilobytes and that takes a little over five maybe six seconds to download on 3g and maybe one to one point five seconds on decent Wi-Fi and That embed parameter is interesting because if you look at a resource in the API The posts the terms the users they're all separate This is something that sometimes frustrates people when they're looking for that one bucket that holds everything But the way that the rest API works in the way that many APIs online work is that they separate resources And those resources have links. So in this case, there's a link where The author of a resource has its own address and you fetch the author from that address in a second request And what in bed does and the reason that everybody? Uses it move this up a little The reason that everybody uses in bed is that it takes that extra resource and it sends it to you in one nice single HTTP request So going back to our example, that's what we're doing here We're getting the 10 most recent posts and we're getting every single term taxonomy author Featured image every other object that is linked to those posts It sounds really good, but let's think about it. Let's compare that to how we build a theme in PHP We're asking WordPress give me everything that you know about and then I'm only going to use the excerpt the date That's very inefficient Fields is the thing that I most regret not launching the rest API with There's reasons why we held off and waited until a later release to add this parameter But there is a parameter underscore fields Which you can add to the query string for any rest API endpoint and This will restrict the amount of data that is sent back to the user So if you take a pages response and you say I want the id link title and excerpt You get only those things and you don't get anything else This lets us fetch only what we need We only send and download the information that's necessary for our application and we as the developers know what that's going to be The controllers in core actually don't generate the data that we don't need and that makes it much much cheaper to compute Which can actually save hosting costs in addition to saving battery on the devices that are downloading your data The way that that works in core and one of the first takeaways for any plug-in developers in the room is That we have a function rest is field included. We have core code built into the rest API controllers Which lets us understand what fields have been requested and then we have this utility function Which will compare a nested string to the list of fields that the user has requested and it understands things like well Maybe I wanted content raw because I am rendering the block editor But I don't need the rendered version because the block editor edit screen is going to do that for us So let's skip computing all of the content. We don't have to generate dynamic blocks We don't have to run any filters on the content We simply skip it because it wasn't asked for If we apply this fields property to our example, we have the same posts endpoint We're still embedding, but now we're restricting the data to only the fields that we actually need to use to render it out This brings our payload down from 160 kilobytes to 60 kilobytes a 60% improvement and It makes it take about half as long We're beginning to see a lot of improvement That's going to be beneficial for both our users and our server So take away number one Please consider doing this when you're writing your own endpoints support fields so that your developers Anyone that's using your API or if you're writing something that gets the data consider only asking for what you need So that you're going to be as That's weird sorry about that got distracted by the search box So that your data is coming back to the clients as fast as possible Now now that we have only the fields that we need let's go back to that embedding It sounds very appealing to say give me all of the related resources that I don't have to make additional requests But we don't use a lot of what comes back. There's a comments that get embedded There's information about other resources. Maybe a plug-in has associated some other resource with posts And so when you're embedding you can specify a list of Relationships relations author WP term, which is one of the custom ones that we define replies Or we can use fields to filter that as well the syntax on the top is a little bit easier Now if we apply this and say I still want my posts, but now I only want the author term in the media embeds We actually do continue to save additional data. We've lost another 16 or 18 kilobytes and we're now at about 75 percent less data than we started with But you'll notice it's actually not much faster than it was before and And the reason for this is that embedding itself is very inefficient It's duplicative if I am getting the data for my own personal blog I am going to be the author for all of those posts in the Jason in every single post it will say Author K. Adam white over and over and over again if we G zip that it can be sent very efficiently But it still has to be decoded and parsed and queried again and again there's also no way to limit the fields on an embedded response the way we do on the posts themselves and If you have a high-scale site, you're gonna have an SEO plug-in You're gonna have an internationalization plug-in these all add data to rest resources And so you might be taking away a few kilobytes of data by using fields at the top level and then adding another megabyte of data by embedding the wrong resource so The other final and probably most important issue with trying to get everything in one big bucket is what happens when things go wrong Because as I said you can end up with responses that have very very big data Embedded within them and that can cause timeouts that can cause errors and that can cause out-of-memory exceptions And if that doesn't work you just have to try it again from the start you have to start over from zero We can do a lot better than this by using the API in a slightly different way Everybody is scared of this we don't like HTTP requests. We try to concatenate our scripts We have trained ourselves over the past decades so well to want to minimize the number of times we ask the server for something But HTTP has improved our servers have improved Many things have changed that make it much more efficient these days to try to get over our fear of Sending multiple requests There's a gist that I've put up at that URL and these slides I've tweeted out the link to them where I walk through this actual example, but Maybe we can get only those posts. We don't need to embed at all We'll get only the data that we need and then we'll look at what categories are used what tags are used We'll build a list of only the IDs for each of those resources that we need on our page then We'll send requests to the categories endpoint and the tags endpoint saying I want my users But please only give me these two or I Need these tags, but all three of these posts are all in the same category word camp Asia So I only need that once If we do this We actually have a situation where it maybe takes two Three four hundred milliseconds to get that initial set of posts But then we have all of the information we need to start all of those additional requests in parallel at the same time and depending on your infrastructure, they can all run and complete very very quickly and Your application only has to wait for whichever of them takes the longest So if you have a bunch of tags, but only one category and only one author and you fire off all those requests at once You're gonna have most of that data ready to go and then as soon as those tags click into place at the end You have the information you just render In this example when I was doing testing prior to this talk It gets things down to 500 300 milliseconds Fractions of that one and a half seconds on fast Wi-Fi that we started with If you're getting a 60 to 80 percent speed boost and having to generate Encode send decode parse and render only 3% of the original data That is a huge huge performance win for every part of your application And it ends up rendering exactly the same screen. We wanted to have at the beginning if We're writing that PHP theme. We just call get the thumbnail. We don't say Please give me the entire dump of every unit associated with this post in the database and then I'll take the one we want There's no reason why we should continue using the API in a different way than we've learned We should be using all of the other core interfaces and WordPress so Takeaway number two Consider it. There's going to be some situations where the application that you're building or the environment where it's running Meek makes having a number of small requests more difficult But consider it test it benchmark it Tools can help make this easier to one of the frequent complaints that I hear as people hear me say things like this and say But that's a lot more to manage I don't want to have to write a middleware layer to assemble all of these pieces together and We actually already have one in Gutenberg. There's core data selectors that are designed to hit and target individual endpoints There is some base functionality in get entity record, which is a very low-level selector within the block editor Which is capable of understanding and processing fields. We can expand this we can make these Foundation more smart and able to work more for us and even simply by using the block editors data selectors out of the box We're going to be doing things mostly in the right way a Related concept briefly is to try to avoid per page 100 We all want to get all of the data for application And we want to know how many we want to be able to render out all those things very quickly when the users clicking through results But it is almost always faster to request multiple pages of data and assemble them than to get one giant request And also if you start combining this with embedding that is an absolute recipe for out-of-memory errors and timeouts Talking about profiling who's familiar with query monitor All right, that's a very small percentage of the hands in the room So everybody else this is the one if I can be generous enough to offer us homework Please download and install query monitor it is a plug-in by John Blackburn who I'm really honored to work with human made and It is the de facto wordpress performance tool it gives us the ability to look at all of the templates that load all of the database queries all of the cache hits on a page and Get reports about what's slow and understand why And for a year and a half we've been able to do that for rest endpoints. I actually didn't know about this until last year I'm still really really excited that we finally can do this, but if we Authenticate a rest API request because you don't want everyone on the planet to be able to understand the performance of your application And then use this envelope parameter Which says to give you additional data about the headers Suddenly we have a wealth of information About cache hits database queries how many database queries which ones were slow All of this information is exposed on every single rest API endpoint if you ask for it And that lets us look through Things we can do the profiling I was doing earlier where we simply use a stopwatch or the developer tools to time How long rendering a page takes But then if you won't realize something slow You no longer have to just accept that you can actually dive in and you can learn exactly what's going on And you can fix it so between Trying to acknowledge ways that the rest api is built to handle parameters like fields There's been a couple other times so far and it was running theme yesterday a contributor day around Understanding how core does things and why There's some things in the core rest api that we wrote in a wordpress way In the derogatory sense. We didn't use namespaces There are certain types of code that we could have tried and we could have used in core if we had been writing it from the ground up In the php7 php8 era We built it six years ago now over that in fact and There's some things that we probably should just accept Well, this looks like the rest of wordpress But there's also a lot of concepts in there that are very valuable fields being one The other and the thing that i'd like to use the remainder of my time to talk about is schema If you're not familiar schema is machine readable documentation. It's also human readable, but it's more intended for the machines There's a standard called jason schema, which we follow in the rest api And this is actually one of the areas where I do think that our documentation in the rest api handbook is quite good. So excuse me If you do go to developer.wordpress.org Slash rest api. There is a page about schema. There's actually also a page about embedding and linking which I would recommend Plug-in developers consider reading through But the schema is a description of the data that can occur in a particular Endpoint a particular meta field or a particular Registered rest field And the difference between register meta and register rest field is another talk which I hope to be able to see somebody else give some time but The way that schema works with all of these is very consistent You give a schema property when you're registering your route or when you're registering your meta field And you tell it what the type of your data is going to be You can tell it what the defaults are going to be You can tell it that you always should expect an integer You can tell it that this will be either a bool or a string Maybe it you know the same way we would annotate a function. This returns an array of items or it returns null if there's an error By writing out our schema. It simplifies the validation. We can take our application and we can build Things assuming that the data is going to be the type that we want Consider how many times we take a string and we check to make sure that it's a numeric string And then we convert it from a string to an integer And then we pass that integer into a function and then when the result of that function comes back And we have to make sure the array key is set We spend a tremendous amount of our time in wordpress coercing values changing data changing the types of things And the rest api is built to make sure that things come to you in the format that you want And the format that you always expect if you use a schema It's also something where you can access the schema for anything if you go to wp jason What you're seeing is the schema for the entire rest api And you can get that for any individual endpoint for any method That lets us get access to it on the client. We can give it to our block editor plugin We can give it to our static site generator and we can say Here is what this data is going to be shaped as and there's also tools. There's a javascript library called ajv Which let us validate that data So if we're building something on the client if we're taking in data and rendering Form that users will use to give us information that we're going to send to the rest api We can check to make sure that all of the types in our object are correct before we send them And then we can show errors immediately to the user if they need to change something without having to wait for an error from the api There's also work That timothy jacob's who's the current one of the lead maintainers of the rest api has been doing around Generating interfaces based on schema if you have a fields called priority And you know that it is going to be a string that is one of several values You are able to write code that says Let me take that my application in the block editor sidebar is going to read that schema And automatically know that because the type is an enum a list of values It renders a drop-down You don't have to write that drop-down. You don't have to hard code those values Although I think you probably should Register it in a way where it has internationalized labels But there's slides here that are linked both in the url at the bottom and in the qr code on the side from word camp new york From a talk timothy gave about this. It's a very very powerful pattern for plugin developers Because if you're building a framework for other engineers Take something like acf They know what shape They can provide you a form to create But then if they do that in a way where if you make a text box That gets registered as a text field on a schema Then they can use that same information to give you a consistent interface across it in different places I'm not actually sure if acf takes advantage of this, but it's a very powerful Concept for anyone that is building something for other developers to then build on top of So we could still continue to manually check our types. We could write custom validation code. We can Return cast to int if is numeric string one more time Or we can take a little bit of time to read the documentation around these schema documents Learn how to write them and add them to our application in a way where it's going to not just make our lives easier But also be a first line of defense against the api receiving garbage data from other people We do like to be able to just automatically trust wordpress to reject bad things And it's obviously only the first line of security to have a schema But it does assist with your validation and with your sanitization in the in custom endpoints Final concept for plug-in developers if you are taking the time to register a schema is to cache it Because you have to compute that object that jason document that you send back And when we first launched the api We realized quite quickly. I think it was actually alan from next door who flagged the ticket That we were doing this over and over again, and it was causing A lot of slowdown. We were repeatedly creating the same object every time we were rendering a bunch of posts in a response and by simply taking The generated value and storing it on the controller as this schema Then we can return it once it's been generated once without having to continually recompute it So if you're building an endpoint for WooCommerce or a translation plug-in or anything else You're able to Generate the schema once store it in a way where you can get access to it over and over again And when we did this in core there were certain endpoints that got 40 faster No, just moved it up so What have we covered today? We've talked about a couple key takeaways around patterns that we should follow if we're fetching data composing multiple requests Building our own endpoints in a way that they support the fields property by using the rest is field included Understanding that we can make multiple requests to linked data to be getting that same information more efficiently And then we can use query monitor to profile all of these requests and make sure that things are working the way that we expect We can also write schema when registering our routes to make our validation our ui generation And a certain degree of our expectations around the security of requests. We receive more reliable The running theme is that there are some things in WordPress core that we find to be pretty Useful patterns and have really been excited to see them take off in a lot of the big plugins in the space There's also a lot of things that we do in WordPress core that I would like to see more plugins Do we're having a very interesting conversation right now with the WooCommerce team about Improving the way that fields is supported on their endpoints because if you're running a big e-commerce site you want to be able to get that information quite quickly And there's certain scaling problems that we've run into on some of our projects It's an area where we can hold each other to account if you're building a plugin for a clients If you're using a plugin that you've gotten from the plugin WordPress.org site See whether or not things work the way you expect if you add fields Maybe it's a smaller response, but is it returning in the right amount of time? That's an opportunity for us to work together to collaborate and to make our tools able to increase performance The performance team within the WordPress core project is doing some incredible work But I think there's an opportunity for us to step up as plugin authors And I I do actually spend most of my time building agency sites rather than working on core So I'm certainly committing to this myself as well To write our own code in a way that's going to be as efficient and as secure as possible And There's a lot of topics we didn't get to cover today. So do please catch me in the hallway track I'd love to talk about some of the confusion that we've had around context over the years I'd love to Explain some of the things we found when you're on a website with 500 000 posts that you expect to work And then suddenly that tags drop down just Doesn't work What tells what puzzles you what would you like to know about about why the rest api is like that? If you can raise your voice either here at the conference or online Then between the developer teams the document documentation teams and the community as a whole we can improve the resources that we provide around Building these tools in the patterns that they were designed to be used And I think we're right at time, but if you'll permit me one final thought I'm very guilty of talking about the rest api a lot It's something that I've had very identified with me personally I've been honored to work with a lot of the people that were the lead developers of it when it was first created And I'm realizing that probably the degree to which we've talked about it as its own special thing has hurt The community as a whole to understand use and feel like we own it So just as yesterday I asked the organizers to switch the talk Track of this conference presentation that i'm giving today from rest api to general development We have a lot of skills in the wordpress community. We do have very high level understanding of rest api best practices php best practices css best practices I think that we should be out there sharing those things outside of our community in those very special modes There's very specific domains But I also think that I want to do a better job of not talking about rest api best practices Not talking about rest api performance, but calling it what it is It's block editor best practices. It's block editor performance That's the foundation that we need to have to be able to work together as a developer community to build the next couple phases of the Block editor and to do it in a way that's going to make the web a lot better for all of the people that rely on this program Thank you