 Okay. So hello. Today we're here to talk about how to create complex fill widgets on Drupal. I'm pretty happy to be here. I'm pretty happy to have all these people here too. This is my first time talking in the background like in sessions. So I'm pretty excited about all this. So thanks for coming. Welcome. This is a link to the slides. You can see it in the top in the bottom right. It is going to be there for around three slides, but if you can get it, I'm going to put it in the chat at the end of the talk. But yeah, welcome. So who am I? Hi, my name is Darisa Spinach. You can find me at like Darisa in almost everywhere. I am from Costa Rica. I live in the Pacific Coast. I live like one block away from the beach, but I never go, but I love it. And I have almost a zoo in my house, so many dogs and cats. And you'll see here, my friend is gone. He usually comes in when I'm talking and start biting my headphones. So if you see that, it, you know, it is that we can't do anything. Yeah. Okay. I'm a Drupal developer. I work as a Drupal backend developer in Siamati, which is a Costa Rican company. And we are an agile full service web agency with a human first collaborative approach in design and excellence in delivery. We are a group of passionate people who wants to make the web a better place. And we really focus on humans and experiences. And that's what we love to do. I've been working with Drupal for about, I think almost six years now. And a lot of that time has been working with Deegan. Deegan is a community driven free and open source open data platform that gives organizations and individuals ultimate freedom to polish and consume structured information. So when I'm talking about Deegan, well, a part of it being a big part of what I do, also this stuff comes from that it was a problem that we were facing in the Deegan new version. And we found out that the best way to achieve it and to solve the problem was actually creating a custom with it. So this is what this, that is where this comes from. So what are we going to cover today? First, just for setting the base, we're going to talk about things we need to know before starting to create widgets, which are fields in Drupal, when there are arrays and annotations, then we will go full hands on on custom widgets, where are they, when do we need them, when do, when we don't need them. And then I'm going to tell you the story of the problem we're having and how we solve it. So let's go. First, I assume some of you already know some of these steps, so I will try to cover it quickly. But yeah, I believe it is, it is good to have like a common base, just in case. So things we need to know, fields. I know we've all, everybody that works with Drupal have interacted with fields. So they are individual components that make every content type in Drupal unique, right? So for example, in, in, in most, in all the content types, we have the body field in a title field, and we can add as many photos we want, we can have an address and email and phone fields, we can have tags, we can have anything, right? And all this stuff is basically powered by a big part of Drupal, which is the field API. This field API is what is in charge of defining all these components of Drupal. The field API is made of three main components, which are the field types, field widgets, and a field formatters. At the beginning, it's a little bit hard to find a difference, but in practice, it is just like the field types is what allows us to define the type of data that is going to be stored. So that way we can say, okay, this is going to be a string, this is going to be a number, this is going to be an entity reference, this is going to be a date, and like that. The field widget is actually how the field is going to be rendered in the form, right? So how is it that the user is going to interact with Drupal to be able to save something in the database? And then the field format, which is how the field is actually rendered for the final user. So widgets and formaries, at the beginning, even for me, it was a little bit weird. I didn't understand, but it is basically that. Widgets is for forms, formaries for the final presentation of the items. So for example, in the case of the text, which we can see in the screenshot, we have an autocomplete filter for adding text in there, and then we have the formaries, which is how we are going to present it. So for example, with the text, we have different ways of showing rendered entities. We can show it as plain text, we can show it as links, and those are formaries. So the main thing here is that the three items that we're talking about, they're all plugins. And for defining each of them, we need to create a class. Each of these classes is going to have annotations, which is what helps Drupal discover the plugins. And also, these classes should return a render array in order to render the item as is. So this is why we're going to cover these two items. So first, render arrays. What are they? They are basically the building blocks of every single Drupal page. Okay? They give us information about the data that we want to present and how we shall present it. They give us hints about that. So in order for Drupal to be able to know how it's going to be the rendering, we need to give them this kind of arrays. This rendering basically means, how are we going to transform from arrays to HTML markup? So how does a render array look like? It'll be something like this. We simply have an array which is an associative thing with keys and values. And here we can see the content or data and the properties. So the content can be there as nested arrays. And the properties are those elements which has the hashtag at the beginning. And that is what tells Drupal or what gives Drupal the hints about how this content shall be displayed. So that's it. There's a lot more to say about render arrays, but you can look at it in a Drupal implementation. And this is like the basis. So the annotations. Every class, every plugin class shall have an annotation. And these annotations are basically used for registering a plugin and describing the metadata of those plugins. So they're basically structured comments which give information about the class that we are creating. So this is how an annotation will look like. You can see the first element is actually just a comment, like the description of the plugin we are creating. And the actual annotation starts like in line four, I think, when we see fill widget. So usually we have in the type of the plugin. And in this case, this will be a full widget. I took this example from the Boolean checkbox widget which is in core. And yeah, basically it has some keys and some values. The annotation syntax is using doctrine. But even then, we have to follow some Drupal standards. So for example, at the end of each value, we need to have a new line. And when we have arrays, we need to keep adding the trailing comments at the end and that kind of stuff. But it is basically doctrine. We'll go deeper in the description of the fill widget later on. Also, if you have any question, you can type it in the chat and I can answer it. So, yeah, now that we have talked about all these three things, we can go deeper into custom widgets. We know that a lot of things in Drupal are made via the UI, but sometimes what Drupal gives us is not enough. So it is then when we have to try to see if we should create something custom. In this case, we're talking about custom widgets. They are, of course, creating in code. And it is interesting because they can be a combination of other existing arrays, I mean existing widgets that we have, but with some extra functionality or maybe they are creating from scratch because what we have is not enough for us and we need to do something completely different. Or maybe it is a mixture of both and we are going to extend an existing widget to add more stuff in it. So custom widgets are very powerful and we can create a lot of stuff with them, though not always are needed. We don't always need to do this. For example, if what we want to do is just display the, alter the display of an element, then probably we don't need to do this. Like if we want to change the label or something like that, we can just use a form alter. If we need to add extra submit functions to our forms, we don't need to do a custom widget for that. We can do it in a form alter. Same with adding extra validation functions or even showing fields conditionally, not necessarily we want, we need to create a full custom widget to do it. In that case, we could do state's API or something. So this takes us to when do we need to do this? Okay. Basically, this is a good question because Drupal writes a lot of widgets and it is pretty hard to find them all out. So the thing is Drupal has a lot of things, but sometimes that is not enough. So we will need to take it, take a breath and try to find out what we need. So I'd say that in order to create a custom widget, we should figure out what is it that our necessity is. For example, I think that storing data in that specific way and make it easy to use and not are prone to users, that specific thing, maybe something we can do with custom widgets. So for example, the classic example of how to do a custom widget will be like the color beaker widget. So why is that a good example? Because we need to save the thing, for example, in an accessible code, but we need the user to put something valid there. So it is easier to have a color beaker and then save the value as the accessible code and that's it. Another example is actually something we had in Deakin. For example, we have some field that we need to save as a URL, but we also need the user to be able to upload a file or link to an external file. We don't have a widget in Drupal that allows us to do that. We have something that allows us to add an URL and we have something that allows us to put a file upload, but not something that allows the user to choose one of the two and save it in the same field. So that's a good example of when a custom widget can be done. Another option is what we're going to do here, which is having multiple fields inside of another field. We're going to go into this realism. So how do we know that what I need to do is not in Drupal? There are so many widgets in Drupal and sometimes it is hard to find existing widgets. I would recommend first checking the core and the country module you have installed. And in order to do this, you can first check Drupal console. With Drupal console, you can just run one command and it will return all deployants that are filled widgets in your current installation, including all the models you have or just the core. So it is fairly easy to know it and you can test and you can see if something about that works. The command will give you the plugin ID and the plugin class so you can go deeper into the code and find out what you need. Even this will help you, this will help you to find out if you need to do some more customizations on the top of one of the existing widgets. And that's pretty cool and it is doable. So there's that. So what was the problem we had in Dekan? Basically, we have one content type which is called data and this content type has just one single field which is called JSON metadata. We did this because we are focusing the schemas. We are schema first. This is a schema-centered platform now and we don't want the users to have to come to the field UI and change the different fields and export all the configuration and everything. So the developers or the people who is in charge of the data can just come to the schema and update the thing and then we just save it in one single field. It is pretty easy to do and easy to handle most now that we are API first. But it is a problem for the users because imagine a user coming here to change the title of the data or changing the modified date or the temporal date or the name of a distribution. I don't know. It will be hard for them. They will need to know JSON in order to do that. So we started using a library called react-json-schema form which did basically what we needed which was actually build a form based on a JSON schema. It was pretty cool but it will still give us some problems. So one of them was we had to build a different UI so it was not so intuitive for users to go there and do the things that they needed to do. And the other thing is we needed more powerful features like the date speaker that library didn't have that or the other element I just told you about the one with the file upload in the URL field and save all in the same URL. So that kind of stuff was being so pretty difficult to do and we say, well, Drupal has the form API and it has a lot of stuff there and it is actually pretty cool and pretty stable and pretty maintained. So why don't we go back to Drupal? And that's what we did. So our solution was to create this JSON form with it. So how do we do this? Basically we need to take five things into account which are where is the class going to be located? Where is the name space it has? Well, where it needs to live? The annotation, the actual class, and the settings. So for the location it is pretty easy. We need to put it under the name of our slash src slash plugin slash field widget and instead of that we should place our class. So in our case the class was called JSON form widget and that's the location where we put it under JSON form widget and all the same items as we have it in the top of the of the page here. I mean, sorry. And that was it. So what was next defining the name space? The name is actually where the class lives. So again, it was the same convention almost. In our case we just had to change the module name to JSON form widget and that's it for our recipe. And what about the annotation? For creating field widgets we need the annotation to be something specific. There are three things that are required in order to have an annotation for a field widget and having Troopa be able to discover the plugin correctly. And those three things are the ID which is basically the machine name, let's say, of the widget that we're going to have. Then the label which is the human name of the element and the field types that this widget supports. So this basically means that if we have a field that is of type string long, we could use this widget which is exactly what we needed. So once we're in this point, we have the class available, right? The class need to implement the widget interface, which is like the interface Troopa has for this kind of stuff, but also Troopa will give us the widget base class which is a base thing that already has a lot of things set up and we can just extend from that and override what we need. So in order to create our widget we just need to have the form element. The form element is the required thing and it is also where the magic happens. It is via render arrays you can add here whatever you want and build a farm. So basically this is how it will look like and instead of this bit is where you will get as creative as you want. So you just need to build the render array and return it here. For example, the string text fill widget only has one element and this is the basic thing, right? So this looks like this and that's it. But we can also have multiple elements inside of our render array. So we could have this text fill and then also another thing which is going to be a file upload and then another thing that is going to be a date beater and then we will need to process all that to become one single thing that is the thing that we're storing. So we're going to look at this. Pretty soon I promise. In order to have the settings form this is an extra setting which is schema because we want the users to be able to define what is the schema that is going to be used to build the thing. So for example we used to have a schema for adjacent schema for data sets and another adjacent schema for publishers and at the end the content type is almost the same. It just has one string-long field that has JSON metadata in it but it's actually different because the data sets will have some fields and the publishers will have other fields but depending on the open data schema where they will be different. So this is what we're accounting for and what do we need to do? There are three things. First we need to set the default values for the schema then actually create the config schema and finally at the form. So for setting default values it is just a matter of using implementing actually the default settings method and here we just need to return the name of the properties or the configurations we need to have. In my case it was just something called schema and also return this thing with the parent default settings. Why is this? Just because if you're extending another widget that has already some settings and you need to append other settings then this is the best way to have both things there and not miss what the parent class gives you. So that's one thing. After doing this we need to create a config schema and this config schema is basically a file inside of the folder config, inside of that folder called schema and in there the file. So this is what the content of my file was. It just needed to be filled with settings and the name of the module to say it is that and then I'm hoping for the schema we were creating. Once we have this we need to actually add the form and again this is just a matter of implementing the settings form method and in here we just need to add again a render array displaying, giving Drupal the hints about what it needs to display. So in this case my schema field in the settings form for this widget is going to be just a text field where people is going to put the name of the schema. In my case we already have the schemas defined somewhere else and so I just needed the people to put their the name of the schema that we are going to retrieve and using a service where we're treating that but if you want to do something like this like you could put there a string long again and have a text area and put the whole schema inside of that that could work too or get creative. It's fine. So yeah this is what we did and at this point when you go to the managed form display for the data set I mean for the content type you want to apply this to. This is what you will see now you have the widget settings form and the field for the configuration. So people should put here whatever schema name it is and we will retrieve that schema and build a form based on it. So yeah what happens if you close the element if you just put the name there and click on update you wouldn't see what you put in there so there's an extra thing which is adding the settings summary. If you want to have a setting summary you just need to implement the setting summary method which again is just a random array which has the data that you want to display. So this is how it will look like once you close it. Now it has the summary here because the user put in data set so that's what we see now. So for now this has been like simple things and in order to make it complex we need to be creative but basically what we need to do is alter all we want in a form element and build our render array with all the elements we want and then this is the tricky part because probably creating a render array for the form element is is not so hard depending on what we need but then if as in our case you are you need to put there a lot of fields and then save it in just one single thing you need to do some handling of the data and this handling of the data needs to be done using the implementing the method extra form values. This function allows you to take the values from this submitted data process all that thing put it as you need and then Drupal will know what to do with that that that is the actual value of the field you are putting so in our case we wanted to show a full form with a lot of fields and have the users be able to put every single thing in there once they click on submit it should all be saved in just one field which was the JSON metadata so let's say it I'm gonna stop sharing for a second to change um to change what I'm presenting and okay I think you can see my screen now so basically this is what we have this is where we will change the thing and this is the code okay so this is code um here you can see um the location of the things I'm gonna close this a little bit you can see this is where our schema was um created and this is where all the magic happens so we have the names space defined as I told you before we have the annotation here um so as you saw we have the tree required elements which are id label and field types and we're extending it with that base in our case it is like this because we needed to have a completely different thing so and to independence injections because I needed to have another service that we are using and just that you can see here the default settings thing being implemented and the settings form and the settings summary and all that it does is for us to be able to come here look at these JSON metadata which is a string long thing and make it be a JSON form so if we do this we need to change the actual schema in my case I'm going to put your data set I'm gonna update that and I'm gonna save it so when it was the other widget this is how the form looked like I'm sorry I haven't updated my triple side this is about practice please update your sites um but yeah okay so once it is saved if we reload this page we're going to see we have the full form so this is quite easier for users to be able to come here and update each of the things we have so how did I did this basically I just used the form element function in this function what I'm doing is bringing the um default values of of what is like bringing the values of the node what is already saved in case there is something saved um if it is new of course it will be an empty array this default data but basically I just called a function called get JSON form with all the information I needed to build the form with so this get JSON form is what actually does my magic um so here I am basically just retrieving the schema in order to build the form and once I get the schema I am iterating over the properties it has so I'm just going to show the schema real quick well this is a big schema it has a lot of fields so for example it has all these properties it has a proper title title identifier and you can see here that we have information about what those fields should be okay remember that the JSON schemas what what they do is actually help us validate JSON so um if we're already validated in that game stat we are now trying to also build the form based on it so here what I'm doing is actually iterating our those properties and getting the form element necessary for each of them so the form the get form element these these other functions are private because they are not really used anywhere else um but it is what we have right now so this get form element actually takes in the type of the property we were looking at so for example for type the type is a strong for title uh the type is also string the for identifier is also string but for example for for the field um let me find it for the field polisher it is actually an object so the the main elements we have in this schema are objects arrays and strings so I just got different functions for each of these type of items so we can actually print or or specify establish where is the render array for for them in this case um the handle string element what it does is basically set an element that is going to be a type text field so the widget that is that it is using is a text field but then we add a lot of a lot of stuff in based on the other properties of the element so if the property has a title then we append a title to the element if it has a description we attend the description and that's how we we are doing the things for example if you see this element here the frequency it is a string but it has a property called enum enum so what I do is if it has this property then we convert the item to be a select list and that's how this is all being handled um just doing a lot of conditionals checking what is that we need to print and this not only works with this schema it should be able to work with other schemas as well in our case this is these are like the three items that we always have like object arrays and strings but if we are going to build something and that has different types and we can also add it here um I think it's for the issue um so the handling of everything is complex for example how we added the arrays or how we added the objects was kind of interesting so for example I'm going to show you the object thing um where okay so here it is so the function for handling the objects is just a function that has a field it could be a field I forgot the name well a field group something like a field group I forgot the name right now or we can have also details or that kind of um elements so if you see this for example the organization which was an object we have it as a as a details element here a field set just remember it it could also have been a field set um so here we make the field set and then we start calling again the gut form element with the specific properties that the publisher has so it is just a matter of getting deeper and deeper levels of nesting but um we can add the the things the more the most tricky part here was actually with the arrays because we needed to be able to add more elements and remove the last element that was added um so it was it was kind of interesting so in that case for handling arrays um this is what I had to do there's also something interesting here because we needed to handle simple arrays but also complex arrays what are these each of these things the simple arrays are something like the categories like we have just one text field or number or whatever it is but just we add items one by one okay so we add one category and then we add another category and the category is just one property same thing with tags here but what happens if what you have is an array of objects that is going to be a little bit different because you need to um add each element and then call again the things to build the elements that are inside of that object so building the arrays in general was a little bit more confusing because we needed to handle like the different deltas and the different keys because it is a multi-value thing and yeah so what we did for this was basically add some um objects callbacks for each of the items and we needed to um have those callbacks somewhere that Drupal could see them so I found out this gave me a lot of headaches but we found out that um when we wanted to add the callback here it didn't work it wouldn't work actually this one they wouldn't work if they were inside of the form widget or of the custom widget class I'm not really sure about why but if you need to do that inside of um your custom widget the best thing you can do is to have the objects callback to be inside of your class but the submit callback to be outside of it so I just had the submit callback to be in my .module file and in here I picked up the name of the item we were adding and check check that against the triggering element to make sure we are adding the extra field to the right field so for example um when I have category and tags if I click on add one more tag that we actually get the other item here and not here or not get them replaced so in order to do that um I think this is a little bit confusing but in order to do that what I had to do was to set some values in the form state for example here we were adding um the elements and how many elements there were so if we didn't have elements at the beginning because it is we're adding a new element then we just put a value of one to be able to print one field and if we already have more things which is what we do in these other callbacks in the .module we increase the amount of items depending on what was the triggered element triggering element um then we bring that up here and build that amount of elements so that was one of the confusing things and the other um was uh actually processing the data so in our case we had all these fields and we needed all of them to be translated into one JSON so it is then when I had to use the um extract form extract form values which is this this function here and here we just needed to go over each of the items we had based on how we build it and at the end once we had the data the data already built with the elements we needed um we will just translate that to JSON and then set the value to the items and then filter or remove the empty things and finally set the element set the values actually into the right place in a farm state so this was one of the most difficult things but the extract extract form values is what actually led us um process our data and and flatten our data into what we really need to store it in the database so I know this bit is um a little bit confusing but I'm confident that with that um new new items can can be done this work is going to be well it is open source so um I don't know if I put the link here but here this is where where the module lives right now it is part of the beacon but it will soon be something outside so if somebody wants to contribute you're pretty welcome there's a lot of things there's a lot of work to do still and we're open to contributions so yeah um I'm not sure if there is questions about this I know there is but we're almost out of time so in case um you want it you can ping me or talk to me anywhere in any place like I'm in Twitter or I'm also in triple org so you can tell me I'm looking at the question let me see yeah okay so Max is asking if in my case the data is still just getting saved to the database as a single text field and yes that's totally true um it is still a single text field so if you click save here or you go to view what you will get is actually just um once it loads my computer is weird lately you get all the JSON and it is just one single thing so um I know Sean already put the link to it but I will put here also going to the to the module directly because it is in another branch it is not yet in in master branch but you can you can look at it there um again there are still some things that need to be fixed there I'm still working on it but all contributions are welcome and yeah you can find me on on the Drupal Slack too or in the Bethcams Slack as Darisa I think nobody else is Darisa so yeah um during to any page loading issues with the extra form elements for the larger the JSON is okay so um I think we haven't got into big issues because but I think that's mainly because the items are pretty simple I've noticed it gets a little bit slower when we have the Ajax callbacks um but yeah I think that's like the only thing I will I will say is is making it slower we cool yeah it will be a full country Drupal module um someday hopefully soon um right now it is inside of the decan because we are we need some decan parts to make it work for example the the schema retriever but we're working on getting it out and and be something separated um so yeah um I think this is it so thank you so much for being here again keeping contact um I'm happy to talk every every time and yeah thank you so much