 I think I don't need it, but anyway, I'm connected to it. Can I have some more? Yeah, I will bring it. Thank you. We have one more minute? Okay. Hello. Welcome, everybody. Thank you for coming to this session. We will talk about Form API. English is not my mother tongue, so please hope that you understand me properly. If you have any problem, please ask me or stop me, and maybe I'm saying something wrong every time, so you can tell me. Okay, so my name is Ricardo Santh. I'm a Drupal developer and DevOps. You can find me as Tunic on Drupalorg and San Santhe on Twitter. I'm one of the founders of Metadrop. Metadrop is a Drupal strategy and consultancy company based in Spain. We work in challenging projects worldwide. We are very community-oriented and we strongly believe on the open source and its values and what you may find as involved in the ECQs. And we like to face complex problems like the Form API. First, the walkthrough on this session. I'm going to introduce the Form API. Then we'll show you a very simplified workflow of the Form API. Then we will talk about the Form API stages, then about the cache that may say persistence, Ajax requests, and finally we'll see a complete workflow. And then I would like to highlight some key points when we work with the Form API. Well, I guess that all of you know what is Form API. It's just an abstraction of the HTML forms. It allows you to handle submit persistence, altering and rendering forms. It handles things like security that's very important. It handles also Ajax in forms. And also allows to alter forms that already exist in Drupal, created by other models. And of course it takes care of the safety of all the operations. And of course it allows you to create your own custom forms. How is this done? Well, mainly there is some service that handles the forms. The form builder that has many classes involved, like the form builder itself, form validator, form submitter, form base, etc. There are a bunch of them. And its form has a class that implements the form. It extends usually the form base. The class that creates the initial form. And usually you have a form array that's usually called the form. And also we have the form state, the second variable, that's very important during all the life of a form in the Form API workflow. The form array is just a PHP array. It handles the structure and behavior of the form. It's not exactly a render array. It became some form array, but it's not a render array, but it's not a render array at the beginning. And it's available during all form API stages, mainly because it's been changed during the workflow. Then we have the form state. It's usually a form state instance, or at least a form state interface instance. It contains the state of the form. It includes the inputs, the errors, if there are any, or flags like, for example, if the form should be cached. It's a state of the form. We'll see later. It's also available in all form API stages, and of course it's been changed because it's the state of the form. Some notable values that you have to know, it's the form ID. It's like an identification of the form, which form is this one. It's like the class of the form. And the form build ID. It's a specific build of this form. You can think of it like an instance. The form ID is the class, and the form build is the instance of that form. And there is a token that it ensures that the form was previously requested by Drupal. It's a security measure. So let's see a simplified form API workflow diagram. It's very simple. Here you have just a user requesting a form. So when the request hits Drupal, Drupal detects that there is a form in that path URL and calls the form build function. It calls the alter hooks so anyone can change the form. And at the end it recursively builds all form elements and returns the form. I guess that it's something that you already know. When the user submits the form, Drupal again builds the form from scratch. This is part of how Drupal handles security. You have to recreate the form to match the form with the information that the user sent to Drupal because, as you know, you never should trust the input from the user. You have to validate and check that everything is okay. So Drupal builds the form again to have the good reference on how the form should be. Once it's built, Drupal validates the form. If there are errors, they are added to the form and returned to the user. If not, the submit handlers are executed and the response is returned to the user. This is like the user gets the form, submits the values and Drupal returns the result. Then we have the rebuild. This is one of the four dynamic forms or forms that change with the user input. In this case, the user sends the input and Drupal again builds the form, validates and executes a handler. Usually, there is something changing the form and state, and this includes the flag that says to Drupal that the form should be rebuilt, and in Drupal, instead of returning anything now, what it does is to build again the form with the form state change. This allows to have different elements. For example, the typical example is when you have multiple field values and you want to add another one. You click on the button to add another element, and then Drupal does is to rebuild the form with another element. Once it's rebuilt, it is sent to the user. You can see what I call the rebuilding loop. The user requests a page with a form. Drupal delivers the first form. The user sends the first input, and then we enter in the rebuilding loop, where if it's not the final submit, Drupal builds again the form, sends to the user. The user sends the form to Drupal, and we start again until the user sends the final submit, and Drupal processes it. During this loop, it's where the form is being changed, adding new elements, removing elements, doing that kind of action on the form. One example is, as I said, adding a new element. We have one field with just one element. Then after a submit and rebuild, another element is added, and then the user wants to add another element, clicks on the add another item, and another item is added to the form. What about form responses? It's important to know what form can answer, let's say, the response width. It's a markup, usually when you end the submission, or a redirection. This is what happens when you create a new node, that you are redirected to the page of the node, and as we have seen, a rebuild form or a form with errors. Really, the submit handler can decide what to return, but the first four options are the most common. Let's say it's that simple. Well, it's not, of course. What you see on the background is the complete workflow illustration, that it's far more complex, because there are many details, like, well, how to handle security, how Drupal keeps the track of the data between requests, how is Ajax handled, and validation, submission, rebuild in detail, and other workflow details, and that's what we are going to see now. I advise you that it's hard to understand on the first site, so don't try to get the illustration, because it's going to be hard. I'll update, sorry, upload the illustration to Drupal.org after the session. Well, in a few days, probably. So take it this more like a reference, okay? It's not something that you will learn in this session, but you will have a reference when you need, when you have some problem with form API, you will be able to come to this illustration and we'll try to figure out what is wrong with your setup. So let's go with the details. The first stage is the retrieve and prepare. It's like a double stage, two parts. The first one is the, the first one of this first stage is retrieve. It's pretty simple. It's just calling the form build function. In this case, the dynamic elements, depending on the form state are added. Obviously in the first call, you get the default form, but when you are rebuilding the form and you come back here, it's when the build function may add more elements or do any dynamic thing that it should do. It's very simple in this stage. The second part of the stage is the prepare. Here is where the build ID is added. Remember, this is like an instance of the form. It's an identification for this particular instance of this form. The security token is added, only for registered users, because it's to avoid cross-site request forgery. So it makes sense with registered users. And here, it's where the alter hooks are called. And it's the first place to customize an existing form. When you are modifying a form, you will hook here. The diagram for this stage is this one. It's, well, you have the get form. It's as if it's on the cache. We will see this later. The form function is called. Sometimes if it returns a response, you get a response to the user. If not, it continues with the next steps, including the form alter. What is the output of this stage? Well, a build form. Remember that it's not a renderer right yet. It's a structured form, but it's completely built. All dynamics elements are in place, and all alter items are in place. And internally, it calls it AMP persist form. You probably won't see this in the documentation, but it's how the code name is. I will say that the names of the form API, the internal names of the function are not very clear, because they reuse some names, and sometimes it's a little bit confusing. But, well, you get used to it when you read a little bit. So this is the AMP persist form. And this is what it's saved to the cache. We will see this later. And this is the end of this stage. Then we have the second stage, the recursive building. In this case, what is done is the goals of this stage is to get a renderable array. So it's something that the render layer of Drupal is able to convert into HTML. It also collects the inputs of the form. It could be the default values, or the ones that the user sent. At the end, those values are the ones that are going to be displayed in the form when the user sees the form, or the ones that are used in the validation and submit function in the next stages. It also detects which button was responsible for triggering the submit, if any, and detects which validate is going to run if we get to the validation stage. Here we have some options to, we'll say, modify or tweak the form. It's not a way to alter the structure but to make some change. Using the process callbacks and the afterbuild callbacks we'll see in short. We have to understand that those are not standard alters. It's, as I said before, it's not for changing the structure in general or the main structure. This stage, you can see the diagram here. As before, I'm not going to stop it. It's not to understand it now. Here we will see that the form state is initialized and you have the recursive process that traverses or works the complete form array, checking all elements, and at the end it decides which are the correct hundreds. About the other options to tweak the form, one is the process callback. It's called top to bottom during this recursive work of the array. It's usually used to expand elements to multiple elements. The typical example is, for example, checkboxes. When you have a checkbox in the form array, you only add one element that is just a form with one property with the different options. When you render this, you need one element for each checkbox. The process function, what it does, is to take this element and expand to multiple elements, one for each checkbox. This is done in many places in Drupal Core. I have count about 100. It's something that you may need, especially if you have custom elements. You may not need it, but it's possible that you may need it. Another example is, for example, to add in JavaScript conditionally, depending on the value of the field. I think it was the search field, that if there is nothing to search, there is some JavaScript that is not added, because it's not needed. The other option to tweak the form is the afterbuild callbacks. This is something that not many people know. It's called bottom to top, so in the way up. In core, it's used less than 10 times, I think. It's unlikely you will need it. One example, for example, is in the translation form. It checks that some files of the Arab related to the language are available. If not, it removes the option. It's good that you know that it exists, but probably you won't need it. About the handler detection is very simple. It depends on the button pressed, or the element that triggered the submission. If there is no button pressed, or it has no handlers, it uses the form level handlers. Here, we only detect. Obviously, the execution is in the next stages, and the next one is the validation stage. It's pretty straightforward. The form is checked. The securities and checks are done, and the values are validated. Here, we check the token. The security token that we have talked before about the check request. Oh. It was the CSF-RF. I don't remember how it was. Request-formerly attack. It validates all the elements from bottom to top, and what it does is calling the validate function if the element has any function. At the end, it validates the form with the validate handler that we have detected in the previous stages. It's important to know that it can change the form in the form state for the next rebuild, because maybe you detect something on the validation that may need a rebuild, and of course, it can enable the rebuild of the form. The diagram covering this stage is this one. As you see, it checked the tokens, and then you have the recursive call on all the elements, including the core validations and the validation on the element and the root element when you are in the root element. About the validation errors, if there is any validation error, they are added to the form. In this case, the submit holders are skipped, and rebuilding is forbidden, because there are errors. You should fix the errors until there is no error, there is no option to rebuild the form. Remember the rebuilding loop that allows you to change the form and add new elements or remove or do any change on the form. If everything is okay, we get to the submit stage. The fourth one. And here, it's also very straightforward. The form is executed, running any holders related to the form. But the response is returned if there is anything to return. If not, there is a redirection to the form again, if I'm not wrong. You can redirect or markup. And here also, you can change the form or form state for the rebuilding. This is very common, because it's the place, for example, when you click on the button to add new element in the submit, it's where usually the form state is changed to add a new, to say that we have instead of three, for example, we have four elements. And in the next rebuild, the form will be rebuilt with four elements. The diagram is this one. Well, first we have to decide if we have to process the submit or not. And if yes, we run the submit holders. We clear from the cache because we have run the form, so we don't need it anymore, at least at this point. And if there is a response, we return it. And if not, we continue to the next step. If we have enabled the rebuilding, if not, we return the response. If we have enabled the rebuilding, we get to the last stage, that's a rebuild. And it's kind of, it's redoing what we have done before to allow to create complex forms, like multi-step forms, to add or remove dynamic elements. Also, it's typical that Ajax calls modify forms. But at the end what they are doing is creating dynamic elements or going back and forth in a multi-step form. What it does, the rebuilding is just building, running again the building steps. This means retrieve. So running the build function of the form, preparing it, that's running the outer hooks. And doing the building, it's traversing or working the form, creating the renderable array. And here, of course, is where the previous change on the form and the form state, although this change to be added to the new version of the form that is going to be delivered to the user. This is the diagram. As you see, we first do the retrieve form, then enable catching. We will talk about catching there in the next slide, because it's a special case. Prepare the form and build it, build it. The cache, I would say persistence, because it's not exactly a cache, because usually the data in the cache is something that you can recover if you lost the data. That's the idea. But in this case, it's data that you may not be able to recover because if the user did some change on the form and you lost the change of the user, there is no way that you can recover what the user did unless the user repeats the steps. So I would say that it's the persistence mechanism of the forms. It allows, again, for complex forms like Ajax, MutiStep and dynamic elements. The good thing is that usually Drupal takes care of it. It's out-enabled when needed, but in some cases you may need to enable it. What it does is to save the form and the form state. It's important to note that the form state only has said given that properties are saved. You can check the getCacheable array to see which ones are. But really what you have to know is that if you want to save the value in the form state, you have to use the crude functions that are very simple, just get set and check if that value exists and Drupal takes care of it. Previously, I don't remember if it was in Drupal 6 or 7, people used the form array and slightly different form, but now I think that it's just using the form state. And about Ajax requests, they are usually something complex when you start to play with them or it seems complex, but at the end, they follow the same, almost the same workflow as the non-Ajax requests. It's important that what they do, it's at the end when you have to return a response to the user, they call the Ajax callback function that you have defined in your form and that function is what collects what you need to return to the user. It's also important to know that the Ajax callbacks can return Ajax commands. If you don't know them, they are pretty powerful, but what you can do is return actions to the front end and the front end may do creating new elements on the HTML elements or adding a new message or you can do a lot of things. It's very interesting. And about how to do an Ajax request, well, there is an element that triggers the Ajax request, then Drupal builds the form again, evaluates and executes it. During the suit meet, usually there are some changes in the form state and the rebuild is enabled. During that rebuild, those values that have been changed makes the form get a... well, new element or remove something or do anything on the form. And at the end of that part you will have the complete form build. In this case is when the Ajax callback enters and returns what part you should return to the user. This is important. The Ajax callback only should return information to the user, not changing the form. And when the response is returned to the Drupal, the JavaScript layer is able to take it and process it. Remember the Ajax command. This layer is taking care of the commands and doing anything on the front that it's needed. Important about the Ajax requests, some things that you should don't do, don't change the form during the Ajax callback. I have seen a lot of people struggling with this because it's true that Ajax callback sounds like, okay, this is the function that do what I need to do on the form when that button is clicked, but it's not. You should do in the submit, okay? The Ajax callback what you should do is to collect the results. And it's true that in some parts of the Drupal documentation this is not clear. So take care with this. It's important. Ajax callback return values don't change things. And well, that's, yeah, I think that's the most important thing that you have to remember about this. And now this is the complete workflow of illustration. This is, I did this for the Drupal 7, I think 8 or 10 years ago. Drupal Evolve, it's quite similar, but it's not the same. This is it. It's, as you see, it's, well, not easy to get all the details in just one first site. I promise it will be available in short because, I mean, the session could be more or less enlightened, but I think that this is the good output of the session. And I hope that it will help people to understand the form API and create and modify forms without struggling because, yeah, 8 years ago I was suffering a lot to understand what the form API did. And when I finally understood what it did, I decided to create something like this. So, well, I hope you will like it. Some notes about this, there are simplifications. So that diagram is, it doesn't have all the information. For example, the patch in the program forms are not covered. They are not very different, but some edge cases. There are some details that are not shown, of course. For example, how the HTML ID are handled, it's not something that it's in that diagram and it's a bit complex. And I think it's not very important for changing the form, but maybe one day you will have a problem, but it's impossible to add it to the diagram. Some edge cases and some security details and more other details. And some key points about the form API. Forms, the forms build function and the altars are for the big change or for the big setup of the form. It's where really the form is created. The process and after builds callbacks are for adjustment and tweaks. Don't try to do something strange in the process because maybe it's not the place, because you have to take into account that there are some things that have already done on the workflow pipeline. So if you change something in the process and you think that Drupal is going to process it again, it's not going to happen. So process and after builds are for tweaks and adjustment. Use the rebuilding loop for dynamic forms or complex forms. Use the form state to store your own variables. Again, use the Ajax callback to return response, not to change the form. And also check the Drupal examples module. It's great. There are about eight examples I think about the form API, including wizard, multi-step, Ajax and many things. It's very interesting. Not only for the form API, there are many other examples, but it's a module that people tend to forget. But when you are calling Drupal, I think that it's important at least to check it because you may find something interesting. And that's all. Thank you. Yeah, so we have one question from the live Q&A. Can we please have the workflow diagram you presented on a public available file? You said you were going to do that. Yeah, I'm going to upload the file probably to Drupal.org on the documentation as I did with the Drupal 7 diagram. I think that I will do with the SBG because I think it's a good format. If anyone has any concern about that, just ask me. But yes, I mean, it will be definitely available. I mean, I did it for the community, let's say. Yes, it's about refreshing a form with new fields. You said that we cannot use adjust callback. Yeah. In that condition, how paragraph can work then? Well, I don't know exactly how paragraph work, but let me check. What happens? Please take the mic with you so we can have it on the live broadcast. Sorry. Yeah. Right. When you add a new paragraph, if you do, let's say, using Ajax, you send a request. It goes, I mean, the form is built. Drupal does any check. And I don't know exactly how paragraph does, but I guess that what it does is when the submit handle is run, it changes the form state, saying in this particular field that there are two paragraphs, there are not two, there are three. And it's that type or whatever. And at that point, the rebuild is enabled. So we go this way to the rebuild pipeline. The form is revealed again. Let's include calling the building function. In this case, the building function detects that there are not two, but three forms, sorry, forms, paragraphs. And the form structure is added with the new paragraph. And then it's rendered. Yeah, rendered. And here we have the Ajax callback. The Ajax callback returns the data to the front end. It's important to say that what you have, I mean, the paragraph is a pretty complex model and the forms are pretty complex. And there's a concept that I didn't want to introduce here, that it's two forms, I think it's called. It's a functionality that Drupal, let's say, prepared to have forms inside forms. I didn't dig too much into it. So I can say much more about this. But I guess that paragraph uses that functionality. Yeah, from what I remember. So in the Ajax, where you put the Ajax callback, you also specify some selector or something. Which part of the form should be replaced by this Ajax callback? So you built a new version of the form, which has the new fields. And then the Ajax callback returns the part of the form that needs to be replaced. And somewhere you have the selector like where on the front end, this should be inserted, this new stuff. So whenever an Ajax callback happens, some part of the form might be replaced. I think the Ajax callback can also do other things than just replacing some stuff. But that's generally how it works, like all the paragraphs and other things that the Ajax, they just built a new version of the form with new fields and everything and then replace a part of it. And you need to be really careful to tell it exactly what you need to replace and you need to put like some wrapper around somewhere. Last time I worked with that, I don't know if there's some new trick that I don't know about, but yeah. Yeah, it's exactly how you have said. But this is something that it's explained in the form API documentation, particularly for the Ajax. There were some problems with the wrapper. There is a change record about that because before it was always wrapped in a div inside Drupal and sent to the front. Now it's changed because it was problematic and now it's supposed to be better. But I can say much more about that story. Any other question? I actually have one. On one of your first slides, you were showing when you added an element, so you have one text element and then you added another one and then you added another one. In each of those steps, you run through the validation, you said. Do you run on there? On each of those steps, you are actually running the validation of the form. So that is why we get, if we have a form which has a required field and you add another item, it actually invalidates the form because you haven't put something in. So this is why it happens. Yeah, exactly. For that problem, you have the limit validation errors that allows you to tell Drupal which fields should be validated because sometimes you don't need to validate the complete form because you are only checking or managing one field in Ajax calls usually. Is that really something we would like to have? Because sometimes the errors can confuse the user. In that case, you should use the limit validation errors. For example, if you want to add a new element, in this case, you want to add a new element, you really don't care about the other values of the form because they are not used for anything. I mean, of course, you can mess all in your submit function because you are accessing a... For example, you say that don't validate this field but your custom code in the submit function access that field. That's something that you shouldn't do. But in a proper way, you would say to Drupal, okay, don't validate anything apart from this field that I'm working on. Then on submit, do whatever you want with that field and then send to the user the only rendered part of that field. I don't know if this answers or... I mean, I can show you... I don't have an example at hand. I can search on Google. But if I don't run, what you do is to just say to Drupal which fields should be validated. It's kind of dangerous because if you only have... I mean, if you put one field, any other field are not validated so you shouldn't touch them, okay? But it's more or less that way. But let's say that it's out of the scope because I was focused on the workflow and the particular uses of, for example, that thing, I don't remember exactly the details so I don't want to say something that is wrong. Sorry. There is one more question for the live Q&A. Which debug methodology and modules are you using to debug Drupal Forms? Well... It's a long time since I didn't debug. But, I mean, apart from Xdebug, I mean, you should use Xdebug when you have to debug something that you don't know what it's doing because Xdebug shows you step-by-step running each line what is happening so I think it's the best way to debug a form. Sorry. I think I step on the cable. Hello. So to debug, using Xdebug and nothing else, I think, maybe if it's something related to one module you should check what tools you have for that module. For example, from Paragraph, I think there are some ways to debug it. But I don't know. Xdebug, sorry. Yeah, if you want to debug forms, okay, you definitely need conditional breakpoints because if you put your breakpoints in the shared code that is called 100 times for different fields, then you need to select by the element name or something and then you can catch exactly what you want instead of having to stop over and over and just see some stuff you don't care about. But you're talking about the functionality of the editor that allows to have... Yeah, normally in the IDE or somewhere where you actually use Xdebug. Do you think that you only do edit to hack the code and you say if this condition is... You can make the condition inside your code by hacking the code. It's like the poor man conditional breakpoint. Yeah, but at some point I didn't know about that so that helped me learn about the conditional breakpoint. It's good. Thank you. Okay, so thank you very much. Thank you. If you want to feedback, I'm glad to hear from you. And remember that there's still some contribution running so if you want, it's always good.