 So how many of you if I say let's science know what I mean No, no many. All right. All right. Cool. So some people went to the party and still showed up first First thing in the morning. Cool. Um, I Am Mateo. I work for Lollabot and I'm here with with Wim He works for for Acquia at the office of the city of Okta and We we are here to talk about the API first initiative. I'm sure that you heard about it Or maybe you came here to find out In any case, we're gonna talk about this. So what we'll say today is that There has been Some effort happening. I would say a lot of effort happening this this past year, especially on the API initiative front and All of that has been directed from Vision and a set of goals that we have on what we think that the Drupal community should be headed in that front and I stress that we think because there needs to be a core conversation on this and People should, you know, just give their opinions. So There are some open challenges and We want to talk about all that but what we value the most of this is the open discussion So we're gonna try to make a big effort for this to be Discussion and not an interview like people going to the mic and asking to the presenters when you go to the mic Please keep in mind that you're asking to the room and Anyone can can answer So about what we want to accomplish There was a Keynote, I think it was a year ago where trees said that He believed in a first-class in first-class web services in Drupal and top-notch APIs out of the box to truly become API first by API first means to to have a focus on exposing your your data via via the API and From there deriving other all the important systems right and The the vision has been to have all the data available In in different in different ways By the way, there's still seats available for those standing in the back So if anybody can raise their hands who still has a seat next to them, so please move forward so that there's room for more people So yeah, the division is basically Drupal is great at structured content It's great at storing data managing data and so on and so it should be possible to get it out in any way That you might need that you might prefer so both restfully and non-restfully Restfully in any format that you might want custom formats even as well Multiple flavors by which I mean the HTTP purist approach that the core REST API offers Or as well the the json api.org so the json api spec approach which is the module that Maté was worked on a lot along with others So those are both restful approaches and there may be additional flavors in the future But also in non-restful ways because sometimes that is the better the better match So via graphql or even things like via couch db for replication and offline first and so on and all of that using any Authentication mechanism so the point is we want you to be able to use your data access your data manipulate your data in any way or form That matches your particular use case Of course, we can't offer all of them right now and it fully stable perfectly documented Perfectly tested away and so on because it's a lot of work But the intent is at least to get to that point where you can use any mechanism you want So the goals are basically Make Drupal 8 stress API usable we achieved in in making making that the case So that is done making Drupal 8 stress API best in class, which is what we're working on nowadays and excellent documentation So that is also something that is in progress because I think Those who few have who have used it have noticed that the documentation isn't as awesome as it should be Ideally, we would also have json api support That is also in progress you ID and revision support because everything is currently ID based Which is not ideal for syncing and so on Revision support is also currently missing. That is something we would like to have but isn't actively being worked on yet Graphql supports. There is a contract module, but it's not fully there yet could have couch db supports That's something that is less frequently requested for now Well to support but but more things as well, but for now these are our high-level goals if you will Why all of these for rest API the rest API in Drupal core the reason is that it supports multiple formats It's not tied to one particular Serialization one particular normalization you can have whatever you want so you can add your own if you want to it supports Any kind of resource not just entities which is also very valuable and it is very configurable So you can customize it very precisely precisely to match your needs Then the other the other one is json api and the goal is for this one to become stable in Drupal 8 3 country up so soon And eventually to end up in Drupal core Why json api because it is an opinionated spec? So it is kind of the opposite of rest API in many ways because it is opinionated. It's possible to have better tooling so libraries frameworks that talk to it directly and so on Precisely thanks to the more opinionated approach that it has which leads to less bike shedding more standardization Therefore you can have that better tooling available Collection support is something that is natively available in json api and for core rest The unfortunate thing is that you have to go and create a view with a rest export display so that you can access it So it is possible to get to access collections in a very precise and curated way But it does take that extra effort Whereas in json api you can just use the URL query string to get just the subset of the collection that you want It's not configurable json api Unlike the core rest API, but it's also an advantage because you don't have to go through the setup process anymore So it just exposes any entity in Drupal core and it just respects the entity access API So it makes things a whole lot simpler by relying on other subsystems and relying less on Configuration which makes the getting started much easier graph ql the goal is for it to become stable and contrib It's a it's similar to json api in many ways But it's perhaps better suited for certain use cases and it's certainly an expectation for for example many react developers So it's really designed for a particular use case And so it makes sense that we also support that for those people who prefer that approach and Excellent documentation Why stable and contrib because it's not just about manually written documentation about how to get started how to set things up and so on It's also about automatically generated documentation that matches your your data model your your formats that are supported and so on along with interactive examples so Being able to go into the documentation drill down to node blog node And then do a get request a patch request and have interactive examples that show you how you can actually achieve those So the starting is too painful right now. So ducks with interactive examples are very important So our current focus is rest api making it best in class. What's missing there are translations modifications of configurations so patching deleting and creating configuration with simple configuration configuration entities file uploads That's a big one that is missing And so I believe personally that without these three things at minimum these three kind of bigger things that are missing We can call ourselves best in class And unfortunately the reason that they're not done yet is because much of this is blocked on other subsystems in Drupal 8 Not providing the right abstractions or the necessary metadata in order to achieve them for example for configuration the lack of Access control handler for certain configuration lack of validation and so on Jason API stable on a 3-contribute and then eventually moving to core what is missing in Jason API mostly at this point because it is already an RC Comprehensive test coverage like the one that we have in Drupal course rest But also everything above everything that is missing in the core STP is also missing in Jason API simply because Again, the subsystems do not provide it yet So we need to fix those things in the Drupal core subsystem So we can have it be working in the core rest API and then we can also make it work in the the Jason API module Graphql the goal there is to become stable and contrib what is missing there currently is active development unfortunately So if people are interested in in graphql or if they're enthusiastic about it Please talk to the graphql maintainers and get it moving forward again or more actively at least and finally Excellent documentation. What is missing there or what has been missing is active development as well That is going to change in the coming weeks and months because we're going to focus on that more So expect big progress there as well as standardization because actually we do have several things already that do these things So we have the rest API doc module Which is something from before the Drupal 8 0 release days even was a proof of concept It works, but it's not complete enough the skimata module which solves another subset of problems to talk some module which Mateo released a not too long ago Which is specifically for Jason API documentation and then there's the open API module like there's lots of different approaches And they all have some overlap the goal is to bring it all together into one standardized approach That works for all the things so that we can Bring all the effort together and not duplicate efforts So what is next after all this short term or long term? Who knows it really depends on how many contributions we get UID and revision support that's important for the syncing use case for example the relax web services country module currently does provide that but only to It's not in core and it's not a comprehensively tested and so on so it would be great to bring that at least partially into Drupal core all of to support so instead of just cookie so session cookie and Basic all support in Drupal core would be great to have all of to authentication Because that is really the expectation that most people have and coach DB support for the offline first type of application That is also provided by the relax web services module in Drupal in contrib But that also might be something we want to move to into Drupal core eventually depending on how how high demand is and so on So with that Hands might back to you So One of the things that I did in order to prepare this presentation was I went through all of the sessions that were accepted at a Drupal con there are lots of them and I kind of try to gather at the ones that That were striking to me that were related to this stuff that we're talking about to kind of I had the impression that there was Very big need of this or at least a very big interest on on this topic And it turns out that we had one training We had sessions in coding and development two sessions there. We had two conversations. We had a Drupal showcase We had two more in the front end space a Bunch of them. I think that count five of them in the horizon struck so in total we have 12 sessions and one training to me that speaks volumes on why Why are there are so many people in in these kind of sessions that there? The need is out there my gut feeling tells me that the multi-distribution channel is one of the biggest reasons why we want to get APIs Whatever the reason is We I think that as a community we need to get there. It's like It's a very important goal because I believe that this is where the future of web applications and the CMS is So how do we get there by collaborating? So this This is a list of people. There's Whim here He's awesome and a part of that He is the initiative coordinator and the rest module sub-coordinator and he's a collaborator in Jason API and he does a lot of other stuff like Making sure that all the things core improves trickle down to things like Jason API and And everything like that and many other things that I'm not saying this just because I want to praise him or every everyone in this slide It is not only about that it is because when you go by home like with the triple-con energy And you want to collaborate if you want to do any of these things there you can reach these people You can reach me if you want to do anything related to Jason API or All authentication or if you are interested in creating automatic documentation Daniel Wiener. He's also very active on the Drupal and in in particular on Jason API and and in rest he knows all about it Then there is that over there. He's he's done a lot of rest work He's kind of leading the effort on the open API effort that Documentation that we was mentioning that we are gonna start pushing forward to that direction. So thanks dad And there's this Damian the serialization maintainer The serialization model is kind of my one of my little triple fetish. It's great And and he is also a risk collaborator. There's so many people I see Chris Chris there Chris has done awesome work in in Jason API. So if you have any desire to collaborate I Believe that all of us are our IOC people. We just jump in triple contribute or even Twitter or This reach and and let's get this started and don't be afraid to report issues good or bad, please report issues And if anyone gets kicked out we can just redo the conversation Yeah, so besides these people there's still many more people people who are actually just getting started in Drupal as well So it is possible to absolutely if you haven't done core contributions yet To get started and help out with novice testing issues and so on I see people being very successful in in helping out that way and getting familiar with how to help out with triple core things So it doesn't need to be as daunting as it may otherwise seem it can also be simpler Think that's what we wanted to say here. So please contact us. Please report issues and so The current state you want to take this one or I think that what we want to In the current state we we are at a place where core rest is in pretty good shape Yeah, I would say like there are some things that kind of from a practical standpoint are Difficult to achieve with core rest. There are some things that from a practical standpoint They're very difficult to achieve without core rest. So using other solutions like JSON API or GraphQL It's not exclusive. We mutually exclusive with using a core rest in one way or another Yeah, and so I forgot to mention that but basically core rests It does allow you to do things that aren't entity related And so it is kind of a JSON API is kind of the sweet spot for everything related to entities But then as soon as you want to do non entity things or completely custom RPC style rest things what rest things Then the rest module is a good fit So core rest is in a good state and we're just maturing it further And so for that we need of course your feedback on which things are getting in a way And of course also we we are making focus into getting JSON JSON API stable and I would say that it is stable We may tack something that we are kind of blocked in some of the core issues to kind of solve the problems from the bottom up and We may not wait for that because we have semantic releases So when that happens, we just bumped into version two and it's an easy upgrade So that seems like a popular solution nowadays and Also one of the things that we wanted to highlight is that we need more of these I don't know if You are familiar with waterwheel it is a suite of a kind of Companion libraries for client-side solutions or that built on top of both core rest and JSON API There's one for JavaScript. There's one for iOS This is great, but we need more we need we need to get things like Starter up for react. We need to to get I know there are many things that people are gonna want to do And we want the tooling that makes things very easy like you want to get started. You don't get the Blue marine theme. No, not anyways that the default theme in Drupal and you're just there kind of we want a good experience in getting a decoupled site or a coupled application or a decoupled Alexa scale or Any of the endless possibilities so these kind of ideas are very useful. They are they may not be 100% Drupal related but This is the whole point. I mean, this is the couple. There are two parts of this, right? Yeah, and some of the outstanding challenges that are like the so hopefully you got a pretty good pretty good view now of where That's where the status of all things related to API first So rest is in a good place Jason API is in a good place Lots of more thing lots more things are in progress There are some things that are missing in core rest and adjacent API, but the vast majority is in a good place Then some of the more interesting remaining challenges are things like image styles How to associate those with with for example an article that has an image field and then how do you associate image style So that you can easily get at them without having to do crazy other requests or hard coding things staying up to date so things like What do you mean by this so stay stay night today it would be like Jason API is very popular now GraphQL is very popular now We can provide these starter kits that I was talking about but then Angular 17 comes out, right? and That's an easy joke So We we need to be able sadly we need to be able to adapt to an ever-changing ecosystem We're doing a good job. I would say we don't need to write the Front wave, but we need to be vigilant in order to stay behind, right? So that is an open challenge Yeah, and things like improving the DX further when you actually do use the rest API or Jason API and things in particular then like If you are patching an entity or posting an entity you get a validation error, but it's currently not in the best of formats So it's like you have to do some parsing kind of That should be better and there are some RFCs some standardization things But nothing is really like there is no consensus on how to do this Things like field aliasing so Drupal of course by default has that field underscore actual field name thing The goal is there to avoid conflict So when Drupal core adds a new base fields to an entity type which we've done recently with the workflow initiative So there is a field specifically to track the moderation status and so on That's why it's there to avoid conflicts there with customly created fields But it does get in the way because it like it feels very droopily to have every single field prefixed with field underscore So we kind of want to find a solution for that, but it's difficult to do so and so on and at the same time it sometimes is at odds with backwards compatibility because we want to provide good error codes for for your requests and if you try to save a node with an NID it may cause 500 error caused by the database library and if we go and fix that and try to provide a 419 error code instead, which would be more appropriate We may be breaking backwards compatibility Because our contract is the output of the API so we cannot change it lightly So those decisions need to be taken seriously So that is also an open challenge in improving the DX the vigilance for not breaking backwards compatibility is very difficult because that sort of thing is kind of an edge case But then maybe some people's code is relying on that and they're detecting at 500 response. So if we fix The API which is to do the right thing to send a 419 we actually break some people's app So it's a it's a slow process and then we need to weigh every single thing very carefully Which is why those kinds of things take some time So yeah, this is not the the full list of course But you can you get a sense of what kinds of problems still exist and with that I think we're and now there's the fun part So So I I said that I didn't want this to feel like an interview So when you have a question you can use both mics, right? Just just lying there and when someone says something or asks a question to the room you can participate by either Standing to the mic or by raising your hand and if you do that Raise one finger to add something to the current topic and raise your whole hand if you want to switch to a different topic So we can kind of keep the discussion organized and if you do participate in the conversation There is candy. I've got candy So clearly Mathew is a really awesome guy here So for those not here Mathew is just laying all the candy and all the fruits on the table in front of us Yeah, let's let's hear it questions frustrations. What works well? What doesn't? Hi, my name is Mariano. I work with Mathew like six months ago with the first version of Shazen API and I would like to say that One thing that it's really helpful for for them is try to contact them when you start a new project For example, we make a newspaper using Shazen API the first alpha versions and Mathew broke the API several times, but for a good reason And that it's it's really really helpful for them because if you are using these modules I contribute like seven patches and I stop to contribute to that module because I didn't use anymore, but I think that kind of contributions make like the ecosystem Bigger because you can explain what are your use cases? and probably the the person that is Coding the module is not using in the way that you expect that it will be used So yeah, we're getting the real-world experiences and the small things that we missed or small oversights or the big oversights Maybe those are the ones we want to hear about. Yeah, thanks Don't forget your candy And this is Peter Just wanted to try to make a connection to a couple other quick conversations where people were very interested in having essentially a small core that could be really a very effective rest back-end or application, you know Feeding an application and just want to throw it out to you into the room Like what is the minimal set of things that really is Drupal core like if I wanted to build You know rest application and manipulate some entities and you know have user accounts essentially What what are the sole? Drupal modules you would leave in core and what what is extraneous to that goal and maybe you guys can Comment if you have any ideas or other people in the room could could mention You know node and user module and obviously the rest API module And system model what else what else would you need right? I think maybe you're asking should we have a Distribution or an install profile that does this out of the box so that it's easier to get on board so that We don't need to say what the list of modules this you can just download and install it so that sure that list of modules I yeah And I think I'm also asking you if you have a sense already of like what is what is the minimal set of things that? You know if you're gonna run a rest server that which are the very fewest number of core models You could have enabled and if you have an idea or someone else does I Totally agree with that idea because I recently had as I also told yesterday I had a personal project where I was doing the rest Back end with a JavaScript application and getting the the Drupal running and getting JavaScript running was very easy But actually finding out which modules served my purpose was very very difficult And I had to spend a lot of time a lot of frustration Just getting a result magic ended up writing all the rest services myself because I couldn't get anything that Suited my needs just because I hadn't found the adjacent API or was that all marginal that could have done it First of all, I'm very sorry Seconds, yes, we we've been talking actually quite a bit with several people and it's been raised again and again That it would be great to have such a rest or API first starter distro or whatever the name would be I think yes It's basically anything related to the entity system all the different field modules so that you can do the rich data modeling That you currently can do minus any of the UI modules So I would think things like well big pipe for example is completely irrelevant for a rest response is that sort of thing if you Want to add something please come ahead So yeah, I agree that it could be much slimmer The one thing I would say is triple core plus Jason API because the vast majority is using entities So Jason API is a key module to make it easier to get started So would it make sense then since this is a core conversation? We have like minimal We have standard would it make sense to all of these things are going into core at some point, right? Like Jason API could go into core so it makes sense to have the install profile there too, right? So yeah, you have the API install profile. That's an out-of-the-box. You just get the API. I very much agree However, there is Like Jason API is not currently in core. No, not currently. Yeah, I know but that's that's the thing We would have to have Jason API first in core in order to have such an install profile in core So maybe that is something we can do with a later point in time I think it's better to for now experiment with a distribution Oh, I'm just saying yeah, yeah, yeah, the distribution definitely definitely is something that we haven't ever had Yes, right you've always had standard you've had minimal you have testing Yeah, but bringing a distro in because it's only core modules sounds like a powerful thing It is roll out the box. Yep. However, there is one concern then with with the Stay up-to-date the evolvability like for now Jason API is popular But maybe in two or three years it isn't anymore and then we are stuck with maintaining this forever, right? Yeah, and then triple nine BC everything must remain BC so we cannot ever remove Jason API like or that's something that there is like It's a double side right there to to sit on that coin the first is that it's very difficult for Contrib module to rely on Jason API How has it that as a dependency? It's harder if it's not in core, right? and also to Dog food and use those API's in core it is impossible if Jason API is not in core But at the same time adding more and more stuff into into core it goes against the principles of the Initial question that Peter was saying. Yeah, exactly. So it's a weighing thing And I think for now the answer is a distribution because that we can do without having to get all the things in core And then we can improve the experience to get started. Yeah, I guess we were even talking more radically. Let's Use composer to build a version of core that is suitable for UI and a version that's suitable for API and Split, you know some a lot of the modules into their own repositories or something So so we we really have a core identified that is essential for all functions and then add on Core modules to build distributions that are useful for different things. Yeah, I think we should explore all that. Yeah, very good Two things since you just mentioned like keeping it in core and eating our own dog food Is is the plan still to eventually down the road start doing the admin interfaces as headless? I mean Big question mark Like who knows maybe if it happens if that's how we if we keep going in that direction, maybe not maybe yes Yeah, I don't think anybody really knows My bets are in the no Triple ten Right. I mean wouldn't that be cool though When you end after this huge effort X months later, you've gained No feature at all. So But it really says for instance of the you know maintainability and and all that but from the user perspective It's hard to yeah, and then second question. I'm still confused that What the development workflow looks like this? So is it gonna be more like rest of us? Or is it gonna be more like restful is? There's a like two vastly different workflows Both of the worlds like the best of both approach. I would like to think Yeah, like there is a the flavor of the Zero configuration that rest of us had like you go enable the module and boom everything's there, right? That was that is something that compared to these triple seven solutions Wasn't there for restful You had to make more an explicit the design in in restful, but at the same time it has many many of the features that you had in restful like the ability of a consumer to decide what they do they want in the response and Built in pagination collections, etc. I think as long as we have that as long as you can Handcraft at times your your endpoints. I think right and that's why I like the serialization module because you can just write a normalizer Hi, I'm Ted. I'm one of the maintainers the open API module and this is sort of like just a call to help I'll probably be sprinting on it on Friday Yep tomorrow, and just if people I've run into a lot of people at the con who know open API swagger outside of Drupal Who've used it on projects where they're maybe connecting to something that provides you with this fire output but basically it's a way for us to document our rest now, but hopefully Json API endpoints through a standard format that we didn't invent and so Which allows us to take that format or to take that output and to use it in a lot of different tools and a lot of Some of the really interesting tools are the ability to make boilerplate code and a lot of different languages But also to make human readable documentation So a sub module of the open API modules open API docs, which uses the swagger UI module to make documentation So that is a JavaScript library. I'm not a JavaScript programmer. So you call for help there It works now, but it probably could work better and potentially women mentioned this that the documentation would actually have like a try it Now feature so you'd say, you know This is the node at node ID input for getting it and you could actually fill everything out Hit try it now and you would see what the output for a particular node is so I think it has a lot of potential to Make Drupal easier for people who don't know Drupal and I think especially for the rest module now Using it you really kind of have to have somebody who knows Drupal because you're gonna get a lot of stuff back in a very Drupal format and We can't really document the rest API in a handbook because it's never gonna be true for any particular site So I think it's important to have that documentation so that we need document it like everybody's API is different Depending on what resources you have and what's a question show of hands who has tried to use rest Documentation and so on on Drupal core, but has been confused frustrated or gotten stuck by it Okay, nope. Okay, so about a dozen hands or how many people found it's good. Maybe maybe yeah, that's a good question One and a half person I think So Friday if anybody's interested or contact me talk to me after the session Yeah, cool. Yeah, I have something to add to that and I think that this is extremely important if we want to have this starter key apps because The probably like Ted was saying I'm not JavaScript developer the people that are most skilled in starting these starter key apps are JavaScript developers, but they are maybe not PHP or Drupal developers. So having sound documentation and easy to install Drupal profiles or Or distributions will will really help on then getting getting productive There was a couple list of a couple modules that could do documentation and we've talked to the rest API doc person He's okay with moving stuff to the open API so he has the thing on his projects a check out open API now and Mateo is the one doing doxen and that's sort of a stop gap until we get something like that, right and There's a module called the schemata module which both rely on that actually does a whole bunch of the heavy lifting for converting normalizing our Type data into the JSON schema format, which is again something we didn't invent which open the API uses But also could be used in other ways Yeah, in other words, lots of things are moving in the direction of standardizing. So that's great So we need more people to help out basically Hello, I just like to first of all say I I'm very excited about this initiative I was didn't I was not aware of it until I came to jupacons jupacons But it resonated with me because I'm a big advocate of like in web development. We have Mobile first, you know some big advocate of that and where about an application development We have I'm big advocate offline first. So this is like the same thing but API So also I do believe that this is where the web is going You know consume your data the most simplest form and then output it however you need it With that my question is and I'll probably get laughed out of the room because it's a very basic question, but I don't But in web probably game in web program in programming languages period. I'm familiar with the concept of First class citizen and up and on here. I know someone your slides. You're mentioning And I'm the API that you work on a best in class And I'm just can you just to find out a little bit for me because I'm not familiar But I don't know if it's the same no, it's basically just saying we have a rest API But there are certain things missing it doesn't match the best API is known on the web like for example I think many people I regard the github API as being a very solid one very documented and so on Basically make the rest API of triple core equally high quality. That's what it was the intent is It's not strictly defined at all It's just like if people think this is a thing that is missing for triple cores rest API to be considered super high quality Like it's it's an example for other communities and other software projects to look up to if there's something missing Let us know and we'll add it to the top priorities list and basically that is it's it's a soft definition Okay, thank you. Yep One of the things that you guys were talking about earlier was adding or changing the API to say enhance error reporting and You're running to a concern of hey if we go ahead and change this we may break backwards compatibility with other people My question is why not do URL versioning? It's a common thing anyone who has built or designed API says run into before is there was their conscious reason that that was Abandoned and purposely, you know ignored Yeah, that type of thing is more breaking the internal API is not the API facing the user but the API facing the other parts of Drupal from the Database abstraction layer and if we break that there may be custom modules that other people are looking Okay, when I get a 500. This is how I'm gonna deal with it But they won't know how to deal with that for 12 or was it for 19 for 19 Unless they know ahead of time that that's coming. Okay Also, we had a note in this in the slides about API versioning. We took it out So hoping that no one will bring up this topic It it is I mean I Cannot see a solution where we have API versioning without having data model versioning and Unfortunately for Drupal that would mean to spin up a new site because you if you delete Let's say if you delete a field you delete it for past notes Like you delete it everywhere you just have one database state at a time and even revisions would not be able to to handle database data model changes and Not being able to do that would corner us into. Okay, let's do versioning work only for minor releases, but that means that you have to be very careful and hack or not hack but intervene into the into the field definitions to Make sure that nothing gets Marked as required because if you mark something as required, you're not deleting anything You are just changing a checkbox, but you're breaking backwards compatibility that requires a major version change and That I mean all that makes it incredibly complicated close to impossible I don't know impossible question mark. Yeah. Well, thank you. Yeah, and just just to add to that API versioning is I think in the grand scheme of things an unsolved problem Nobody has actually solved this you can do URL versioning, but it's also imperfect because Yeah, you could still be overlooking things and so on but there's roughly three ways to to need API versioning to break backwards compatibility and therefore need a new API version first is normalization changes Which is what we're seeing quite a bit in Drupal core right now Like we're missing certain properties of certain fields or certain fields are simply missing from the serialized output Adding those grades. We're fixing things, but at the same time we're breaking backwards compatibility So we would need to issue a new version another is error responses if those are improved to 500 versus 419 example and suddenly my screen savers starts playing and then the third so first is error responses second is Normalization changes and then there's data model changes, which indeed is like there's there's no way to to keep that saying so it's a If you have magical ideas to solve it automatically in the entire world, please come to us You will be rich because the world will pay you a lot to to have your solution, but it's hard This is not a magical solution, but Brian Hirsch mess.gov. Thank you guys so much for working on this Apigee advocates a version of pragmatic rest where you put the version as a prefix right before the resource in the URL and To me that seems like a really sort of convenient easy to understand Solution and I've imagined that versioning data in Drupal, you know, we would just have a prefix before the resource And it hadn't occurred to me that it's like so deeply Complicated like what's the what is the big what is the big hang up? Why is it so complicated? Because for example like a material was explaining if you add a new required fields Like how can you keep both working because the old API version like V zero slash node needs to continue to work the same way But there is actually a new required field sort of fundamentals have changed Which means that posting new data to it cannot be working in a correct way because there is a new required field So anything posted to it must have that required field and only version one is supposed to Throw a validation error version zero is not how do you reconcile those two like that's impossible Same for removing fields. The only way that you can sanely keep your data models the same and not need complex API versioning is if you only ever add New optional fields or remove optional fields and that's basically approach at GraphQL and the likes are taking that way It is solvable But then we would need to somehow limit that in the field UI in Drupal Which means that if you would have enabled the rest module or the json API module at some point before The time when you are still making data model changes We would need to block you make from making such changes because doing so we break the data model So basically the only way we can fix that is by Disallowing making data model changes. Do we want to do that? It's something that we could use as a potential partial solution, but that is where things get so very tricky So one quick follow-up forgive me for being overly simplistic here, but I'd love to know why this wouldn't work I've imagined that you need to support two versions So, you know, you don't break the previous version, but all the previous versions can be broken So let's say I've got version one of some no type and then I come out with version two So isn't it it's basically you need a migration You know, you need to basically roll out a migration script when version two comes out But hopefully you can't create any more of the version ones. You're only accepting new version twos and then Later when they invent version three version one is deprecated and not available You can do that on your Like if you control all the clients then you can do that But if it's an API that is available to other application developers, there is no way to do that But yes, if you control your clients, then yes, you can definitely do that But it's just up not something we can assume in Drupal core So we cannot solve it in an automatic way for everyone, which is why for now. We're not tackling that problem. Thanks I mean one possible solution and this maybe could be done in contrib is if you had some sort of hash of the current state of The data model and you did something like version one is You know version one this is the API for version one. That's the data model And you send along with the request. I'm sitting with version one the data model changes now is version two You have to send that along if you send version one We could at least tell you that hey you are dealing with the old API and it doesn't work because right now if We have no versioning in the data model changes No client would ever know that and they would make requests Maybe sometimes it would work maybe sometimes it wouldn't depending on whether they add a new required field So it seems like almost the best we could do in contrib It would be to say I'm at least going to tell you what I think the version is and the responses would tell me No, it's not you can't do anything which is not great I don't even know if that's worth it, but I seem like that could maybe be possible I would like to say that we could Talk about this a lot and I I invite anyone that wants to talk about today's To to come to the Lullabot booth and probably to the Acquia booth sits there and talk about it like for the rest of the day probably and Over the week But I like to move on to a new topics. Yeah, man. Cool. I'll start a new topic then So first of all, thank you guys just everyone involved with this having done some completely decoupled headless sites already I don't even want to think about what it would have took to do in Drupal 7. So thank you One thing I wanted to add seeing the revision and new you ID support being one of the next phases Is kind of temporary states or composite entity states? Would like say previewing content and that might be a separate concern from this that might mean Maybe changing how previewed content works But when you're editing things in one place and then you have something that needs to be able to load that state to preview and display that It's kind of a challenging thing right now. Yeah, I think the answer is I Think you for preview you don't necessarily need you ID I think the key thing that you're missing right now is a revision support Because you want to be able to show or access a revision that is not the current one You could actually write a a custom rest resource Very easily that does exactly that so it is achievable in that way But I think the thing that will make it much easier is once a workflow initiative matures a bit further Where the content moderation module and the workflows module are actually stable because once those are there We can actually come up with a standardized approach that provides an answer to this So I think we're definitely heading in the direction that it becomes very possible and it's very work-aroundable for now Yeah, very limited with very limited work One of the things with preview is that it sometimes means different things to different people like I will like to do things like in triple seven SPS did to preview my content like be able to put some stuff in draft mode and Schedulate to publish later and go to my site and say okay view my site as one week from today right that would be preview and For that working with revisions Would be just not trivial because it's not just the the entity that you are working on Right. It's everything that relates to that entity needs to kind of follow the same Same revision state timeline or whatever. So it's very tricky. It's nothing to be dismissed and It's possible. I mean I worked for a client that wanted that and we were able to do it by like I said using SPS and Yeah, we're not there yet. I would say preview is definitely the first step. Yeah And that's exactly what we're doing right now is we're instead using draft revisions and Loading those in and previewing that way and maybe I'm not sure honestly how preview works currently But maybe it is making it more dependent on looking at a draft state rather than I Think right now it's more of a temp store, but I might be wrong So you're currently you have implemented a small custom rest resource to access. Yeah, yeah, cool Yeah, if you want to what you could do as a because there are several modules in there in the rest realm If you will that are providing a certain niche that isn't currently standardized in triple core So I would encourage you to publish a module that does exactly what you have right now And makes it available for others to use to contribute to to help expand And maybe it is something that leads or informs leads to a solution in core and informs the solution in core So that it's better. So Helping well standardizing amongst each other in contrib on certain edge cases that aren't in core yet It's a very valuable thing to do as well. It doesn't need to be an enormous thing. It can be very small. Sure. Yeah, thank you I just wanted as we're coming to the end circle back to what Peter was talking about at the beginning you know as a open source developed piece of software We've been having this conversation about the role of core versus what what's consumed and so forth for some time And there's no there's been no real reason for us to re-architect sort of how we are because it's been working But what you're doing feels like to me is giving us a reason to consider the direction that Peter was suggesting Going eventually maybe triple nine or ten or whatever it is in in an architecture that's more in in the philosophy of symphony Where there that were components that are more loosely coupled so you can assemble what it is you need to give the consumer what it needs and That sort of leads us back to where we were we've been in the conversation maybe five years ago But we didn't have a reason to do it But what you guys are doing feels like it's giving us a reason to reconsider the architecture to be more loosely coupled Yep, it could be a logical foundation for the greater Drupal, right? Yeah, exactly And then we're consuming our own in APIs internally not just making APIs for external systems to consume But that's a lot of work. It is a lot of work Rewriting all of Drupal to use all of the API sets, but when we wouldn't do that just to do it because it sounds cool Yeah, we would do it because we had a reason to do it And it feels like to me that you that what you're doing is starting to give us a reason to do that That almost basically stole my question, which was Maybe to put it in a more focused way, you know usually when people talk about API first across Across the whole web world But that usually means that you're gonna you you're consuming your own APIs, right? You're building all your own stuff on the same API you've exposed to everybody else So my question was around like I know it's a long-term vision to do enough re-architecting that we would get to that world inside Of Drupal, but where do you see like what and clearly the the shared API that we would share It doesn't look exactly like JSON API or GraphQL ever. It's something a little lower than that, right? You don't want to go through all the serialization. You don't need HTTP But there's some Semantic primitive there right for like reading and writing of data going through that still goes through the off system Like where do you see like is there is there a kernel of that already? Is it something like is it then is it the entity API? Is there something like that that becomes like one much more Narrow API surface that everything else can build on so you're asking basically the way that Drupal the monolith If you will currently works how it is interacting with it And whether we can change that that's I guess I'm wondering you know Like if you if you imagine that you got to do all the work that would give you a much more Decouple architecture where everything Drupal itself is consuming data through like of one fairly narrow set of API's for reading and writing all of its data Is the current does the kernel of that already exists? Is it something that you know? Like is it something that the entity API for example could become yeah, or is that just not the right place? And it would actually look some at its beat some different layer Yeah I think is a combination of well entity API is kind of the the surface level thing But entity API builds upon several other things the key thing here being the typed data API Which is basically a massive schema of describing all of the the different level So an entity has fields a field as properties and so on and describing all those relationships between the types of each That is how we are able to do graph QL That's how we are able to do rest how we are able to do Jason API because all of those extract their metadata their Foundation their schema from that information So I think we have that I think another part would be the access API at least to some degree because you need access The entity API also has validation constraints that uses the symphony constraint system Which has certain problems, but overall it works So I think those indeed are like entity API plus the things that it builds upon minus some of the things that it integrates with Like for example the render system, which probably like you're not wanting to render html Necessarily for all of this in the world that you're describing Yeah, exactly. So I think removing those bits the html coupled bits kind of Exactly, so it sounds like Like the core of it is there It's just a question of finding all the threads that don't go through those paths where there's other couplings and slowly Snipping those threads and enriching the things that we do have because sometimes we still have Certain pieces of logic that actually belong in some sort of metadata the lower level We have that in forums for example And so removing it from there and embedding it in a way that it can be reused by forums But also Jason API rest and whatever else nice. Thank you. I Would like to add to the people looking at the video at home. You probably want to add the couples from in that side out In in the in the queue later because we had a conversation that was kind of on the lines of that topic and it was It was a very interesting conversation Yeah Hi, my name is lauri my question relates to the same But it's more about from the perspective of contributed modules Like how do we make sure that all of our contributed modules works? I API first as well, especially given that API spectrum is a little bit Fragmented given that we have multiple different like you need to support restful chasing API and craft and craft you L is like the even Get different one. Yep. Are you mostly asking modules that are adding new kinds of structured content? So field types and so on or are you asking modules that provide functionality something like panels is going to be in core but something something like panels or maybe web form or Yeah, basically modules like that not not modules like a big pipe or quit it Yeah, I think that So one of the things we're seeing in the current serialization or normalization gaps that we have in the core rest Module as well as Jason API because it affects Jason API as well Is that certain fields aren't actually? Even though it is in type data So the schema is kind of there, but then certain bits of it are not are ill-defined or not defined or Cause it to not work. We have particularly problems with computed fields. So fixing those gaps To to make make sure that every single thing in type data has a way to be mapped out of PHP slash database storage lands into Normalized Jason or normalized XML or normalized whatever land. I think that is One key thing that we still need to do to make sure that every single thing that in type data can be exposed and then the other thing is that those modules That's the thing that the modules have to do the contra modules is to ensure that they're not doing things in hacky ways or are altering Output or something like that but are instead using type data and are specifically defining the schema the structure of their data correctly and completely including References to other related content like metadata such as such as for example an image field should have the relation with Image style URLs that sort of thing if all that information is there if all that is described then it can be serviced Automatically in rest API in graph QL and so on so think of type data structure Definitions. Yeah, it was also. Yeah, so that was very technical answer. Like I'm sorry. Yeah I was also. Yeah, so It was nice explanation of the technical side, but there's also the whole ecosystem side of this Like how do we make sure that everything in our ecosystem has been taught out in a way that it can be consumed In API first approach, I mean, I don't even know if we do this But I mean some stuff maybe could be better documentation in the code itself. Like I don't know if like the entity Form which extends form-based, but for entities like in the submit valid in the validation Do we put a note there? It said don't put entity validation in your form submit put it in the actual Entity handler stuff like that. Maybe would surface it to Or to explain the difference there because we have a bunch of stuff in a bunch of validation in forms and presumably Config does to which could live with the entity and if developers don't know that Kind of stuff and how would they know if they're not sort of deeply embedded in it Then sir, you're making something that if you if it wasn't any dear exposing via rest that validation It's just not going to happen, but it wouldn't be too hard to make it happen It's just I don't think a lot of good and true developers would know that I think another part of the solution could be Test coverage in a sense that triple core for core rest part of the reason that we are calling it stable and maturing and Maturing further now is that we have comprehensive test coverage So every single entity type will at the moment something like 80 percent But all the key ones that do have full test coverage the more esoteric ones not necessarily yet But that's being worked on But what we're doing in that test coverage is you have to define very little extra code You can just subclass something and then you are exercising an entity in all of the possible way So get patch post to lead exercising validation exercising edge cases and so on and one thing that is currently only partially test is Validation and the completeness of the normalization we could automate more of that so that as soon as you have an entity type And you add another fields type to it for example, then you have Suddenly a test that is failing because you don't have the validation constraints And because you forgot about that in your contract module So I think part of the answer can be that core provides test coverage for all entity types that works automatically And that then fails as soon as you have a contract module that adds something that is not doing things the right way Yeah, that makes that makes sense Also in a Maybe less technical way, I would say that when we go API first we need to Clearly define what we consider the Backend part of Drupal and what is the presentational part of Drupal because we still want to support both, right? And having that line. Well, for instance the layout Module would fall maybe in the presentation part, right because you don't control the layout for all the possible consumers Like does it even make sense to have the concept of layout for an Amazon Echo application?