 Okay, good morning How's everyone doing right now? Good evening last night Not hungover ready to do some intense backhand development So hello For those of you who are not from Montreal, probably you don't know me. My name is Matthew AD I've been a member of the Drupal community here for like 10 years Every year I kind of participate in Drupal camp this presentation I did that Drupal camp Montreal last year and I thought it was really popular Even though I was just winging it I just just fired up the terminal and tried playing around so this year decided I'm gonna clean it up and try to make it Into a real presentation with slides and stuff So I've been working with Drupal for 10 years. I started doing web development when I was in high school and my first job I Was given a bunch of web pages to update and I said no I screwed up I don't want to do that and I looked around on the server and there was php that said great I got myself a book that says Database driven application with php and it said all you can use a database and then I Have to call the guy that went the database and asking for permissions connect with Super-fancy postgres database. That was my first experience building Database driven php applications If you want to reach me, I'll have a slide at the end with the more contact information for now Let's just start with the main subject. Okay, so Let's start with this question How many of you When you first fired up Drupal a spot. Yeah, I know how Drupal works. So if I just look at the database for Drupal 8 I'll know basically how things change from there Yeah, that's right. So how much did you understand from doing that? Yeah, like I gave up after a while, right? Because really Drupal 8 is just not the same paradigm as what I was doing with my postgres when I started Even Drupal 7 it's not a database application anymore. It's like a whole new kind of framework for It's a services and objects application And what that implies is that We really don't care about the database anymore because it's just one of many possible storage engines And just a Drupal entity system will use One of two storage engines depending on what kind of entity we're talking about whether it's fieldable or configurable So it really does matter in this kind of application our interface objects are almost synonymous with interfaces and If we want to understand the Drupal 8 application we have to look at what kind of interfaces are available to us to manipulate that So during this presentation I promise I will not Present a single line or a query of SQL This is banned a bad concept It just confuses us and it doesn't really help us understand what's going on behind We will do all of our thinking using the interface that are available to us And we're going to try to be as solid as we can about it. You don't know what solid is Yes, because we all became experts the object-oriented developers Okay So first a little bit of context the thing with relational databases is that they're it's not dinosaurs and what sharks They they were invented sometimes like the 60s or 70s because Business data is becoming something widespread and problematic and everybody was reinventing the wheel We had Software mixed with data and then people said, you know We can just separate But the tables of data are from the applications and put a nice little query interface in between That query interface became SQL And that was great because now there was a DBA that manage all the business data And you had your programmers on the other side and the programmers just queried things so that the database the database took care of making sure that the data was safe and it wasn't corrupt and You could not bankrupt the multinational corporation for soft And when we started building PHP applications, we thought well, that's great We have all this open-source database software that solves 50% of our application problem We can store data and we can index it and query it and we can just focus on building an application But didn't really work out the way we intended and the proof of that is just how much trouble we had with SQL injection because The structured database it wasn't really designed for a web application It was designed for holding business data and managing it and making it powerful and efficient And that's not our problem here at all. So As the domain of web applications evolved We ended up having to teach junior developers over and over again No, you you cannot just concatenate the string and send it to the database You have to run it through all these filters Otherwise you might bankrupt a multinational corporation and that's that's the problem is you always meant to solve in the first place So clearly we were going in the wrong direction so the smart people and I'm talking like Martin Fowler and uncle Bob and all the agile Object-oriented people they said okay a real enterprise application Needs a pattern for managing the data storage and that kind of went to two different directions Depending on what your philosophy was on one side. There's the active record pattern and the other side There's a data marker pattern So I'm not an expert on the subject I just I just looked at it because I was curious about what kind of system Drupal attended implemented But the most basic one is active record which as I understand it means You write a class that wraps your table and Then you can just add whatever interfaces you want and that's very popular for example Exactly the kind of people who like boogie on rails because it's a basic framework in Ruby but there's also many php implementations of that stuff what's the one and It's called eloquent the one that's in popular php frameworks and You have implementations in every other language and frameworks also and the alternative to that is the data map which says you know Probably the database is some legacy business system and we don't want our application contaminated with that stuff So we're just going to create our ideal objects And then through some other kind of system. We're going to map this ideal object to what the actual storage is So this is the doctrine or m it's Bundled or actually it's like a kind of a partner of symphony. It's the most supportive system in symphony And it lets you just declare what your objects are and then using annotations and say well, you know, we're gonna Generate a table for this and it's going to be called products. It can be called anything And our columns are going to be these and they're got these constraints So it's a very straightforward way of both building application and describing the storage So these orange frameworks, you have a flavor of either of them in every language But they sort of got a bad reputation after a while because people were saying now This is a leaky abstraction as soon as I try to do something that's outside of the framework I have to learn SQL again To find out how to fit that into the framework and the problem basically is the framework and stuff You can't have a one universal system That's going to be able to properly map data to storage in every application. So in cases like this You sort of have to write your own custom built ORM system And that takes us back to Drupal and If you go back in time, you know Web content used to be so simple your page and the page of the title and the body and Maybe some tags and it was very straightforward to just make a table It's yesterday the body columns the title column. Here you go with your data. Now it's your content then somebody thought like hey that's great and all but You know, we can have a hook that can let us create custom fields These custom fields can have custom widgets. So and then someone said, you know writing hooks a lot of trouble. So we're gonna have Nice interface that lets us generate configuration that adds custom fields to stuff And then someone said hey, it's great. It can add that to pages, but hey, I want to add that to users as well and then someone said okay, but What about translations like my language is left to write the bottom to top Like how do I manage that and I want that in a hundred languages and I want to keep revisions of everything And I'm not even going to mention paragraphs in that because that makes it insanely more complicated So what we ended up with is a data domain That is a configurable structured translatable revisionable typed document and I don't know what more needs to be said When I know save it should just work. I don't want to call 15 different methods here So that I'm a purpose isn't going to make it for that We need a way heavier solution okay so just to illustrate the kind of Domain we're working working within Drupal in Drupal Lake at least We have the entity and this entity as like two other dimensions which are it's possible translations and it's possible revisions And within an entity we have two kinds of fields There are bundle generated fields and there are base fields Within a field. We have a list of items. We can have zero or more items of that field for any possible field And within a field item we have properties Sometimes that probably is just called value leading to many confusing Interface calls, but usually you'll have things that are more specific like entity ID for reference or the text filter form that being said To navigate through all of these dimensions and hierarchies we're going to need solid interfaces and Drupal has a lot of these interfaces So just to scan down an object. Here's what we gotta do First on the entity interface. There's the get type data because the actual data of an entity Does not exist on the entity object itself. It is bridged Once we get get type data we get the data of the entity itself as a complex data interface Which is kind of like a key to write But an actual object with more stuff The complex data interface you can call get properties again, and then you get a whole bunch of field item list interface objects with the keys being the field names on the entity You can also Field item list interface This one has a zero or more so you're going with numeric index and then you get your field items And the field item interface you can add the properties again, which gets you the type data interface Which are the individual properties like text format or value? By their keys and then finally once you have your final value object You can call get value on it to get the raw string or number or whatever it actually holds So anything in there and To help us out and confuse us at the same time each of these interfaces Implements a whole bunch of magic methods so we can pretend that we're using them as arrays or objects or Just set them and get them like we did in Drupal 7 and behind the scenes and magic happens It also implements Some more fancy PHP iterator stuff so you can use for each Objects or you can use array access so you can call directly zero instead of get you so Let's go see how that works out in the term this for those who don't know this it is the Syshow which is implemented both in brush and the Drupal console Which lets you do? Interactive debugging of your application So I'm going to create a node the most elementary type of knowledge in the page Zoom in yeah, I checked this morning. Oh, yes at the top. Yeah Okay, so I have my node entity floor So there is no actual data on this the data exists on the Bridge object, which is the type data So to get my fields I have to use the Get fields property So this is my first property and it's equivalent to Get type data Should have all the fields But there will be called properties. There you go. Okay So the entity adapter wraps all the data around the entity So now we have all our fields here. We're going to have a look at the body because It's a classic field. It's called a field item list in this case. This should work also There you go. So our body the field item list What's on this a whole bunch of other interfaces? Can access a fan count delete filter go to the first Lots of magic magic also so Since we're dealing with interfaces. It's always interesting to look at what kind of classes we are dealing with So the body is a field item list interface And its parents are the item list and the type data class The type data class shows up everywhere in the hierarchy. It's both a container and the final contained item So it gives you a pretty universal interface to What your entity object is doing? Another interesting thing to look at is the Interfaces that are implemented So here you have a lot more stuff And it's wrapping because the text is too big but For instance, the traversable type data interface lets you do for each About so in this case, there are no keys because it's like this. I think I'm about zero bad. So nothing's gonna happen Let's try an interface I'm gonna count So we have no items in our box Tricky to start using the interface. Can we create directly? First So I don't have a bucket Yeah, yeah, I'm trying to figure out. How did I create this? I think I can just Go with the value. I'm gonna cheat a little bit. I'm just gonna use a magic set of Arnoir, this is a complex text one Probably shouldn't have done it with a title. It's much more simple. Yeah, that's a field item list And you'll find that This is Posed to give you the clearest line as this Yeah, so this is an example of the magic methods that are available on this interface So I have my title should be able to write Meaning that I've used a magic method to write into the value property of title So if I inspect title more closely My title is a field item list Count it. I have one property I get the property Okay, now. Okay. So my title Is a string item field type And within the field type If I look last part of the title field It uses the map data type the type data and field item base As well as string item base This means I'm not at the end of my hierarchy yet I can for each of the title as a value Okay, so within my title field I also have a string data item The value is not simply a text property. It's another type data object Set that one And it has its own interface So we have to think of this hierarchy as having I don't know four or five levels depending on whether you count iteration And the end is where you actually get your typing because The string value if we put into the title does not Actually help us With the domain modeling that we need to do So everything at the end is wrapped around the final All this data type the data type is the very elementary Object in our iRT Can I go back to the body that still exists? Because I just remembered How to get some stuff to all right that's going to be more interesting So now I have one value which means I can do this Here we go and this Again is a magic method So it should be completely equivalent True, okay Actually, is it identical? So it's the same object. It's all by reference. My body Is a text with summary item. So this body item And the body item kind of interfaces are on it Good strength and to the iterator so we can also iterate on this one So we can call get properties And that's one of the most interesting methods because it tells you What's available on Your field this one is a value of format and a summary If you pass through it'll include computer properties We'll come back to that later In the end though, I just want to inspect or modify What's on my class? I will Focus your attention on this thing Oh, that's something. Oh, sorry Tricked by the magic methods. So the body item if I call format on it It will give me a filter format data type If I just call format it gives me the value that's wrapped around it It's a very common mistake and it's confusing for no reason But the filter format data type has some interesting properties that the string value doesn't have So if I call format right to manipulate my filter format object I can Get Stable options You can have my list Of options that are available for that field. So this is no longer coupled to the Entity form system. It now exists on the object Those are all my possible values So if I go On my format and try to set it I have to call set value To something that's nonsense It will do it But it's no longer part of my options And that takes us to our next topic validation every object and the type data hierarchy whether it's The entity the field item list the field item the elementary data type item as the validate method And this allows us for once from part To decouple all the constraints from storage MySQL and different storage engines usually offer constraints like not null or the string needs to be a certain size But when we have a more complex data model as we do in loophole where we want to manage documents This is not sufficient We need to apply more advanced constraints So we have a validation system It allows us to do that and it also allows us to decouple constraints from forms, which is how it was handled in triple seven If you wanted to uh, for example mark something as an email You had to validate it with an email as part of the form widget This has all been moved onto the entity data object Which means our domain model is decoupled both from storage and ui and ui which has Interesting potentials that we'll look at later concerning Serialization back to the demo so I gave my body feel a nonsensical value Which means I should be able to validate it and I get an object of type constraint violation list Which doesn't tell me much at this point because it will always return a constraint violation I look at the count. It says I have one violation Which is the value selected is not a valid choice Which means my nonsense value was filtered against the options of the field right inside the object Interesting thing about validation Is that it goes back up the tree Which means I can also validate my node. I also get a constraint violation list I have my one error You'll notice one difference Before it says on the object type filter format. My value was not valid Now it says on an object of type entity adapter within the body Within the first item within the format My value is not a valid choice meaning we have a tree of validation errors And this is what the entity form system uses now to validate that the data the content you've entered into the site Is valid or not using this path it will mark the widgets that you entered incorrectly But the validation no longer needs to exist on the form although you can still add some rules But they have to be UI specific So to know what kind of Validation constraints are available You have them on the objects itself Okay If I go back to my Strings here we go and this is actually Called a complex data Now this is a configurable constraints Oh, I'll wait. I use the wrong field. Sorry So the title field As a length constraint of the body Does not have constraints There are two constraints here the primitive type constraint which comes from the text Which means this has to be a text And the allowed values constraints, which means My choice has to be part of the choices that were configured for this one Actually, let's try the other constraints your primitive type Will the format allow me to set a non-stream value? Let's try this So now I have a big time object That's part of my format Which is says right now Wonderful All that didn't work Because it tries to cast a string Let's try something that's more numeric Now it behave better. I tried to do something too weird with it And so that Value accepted is not a valid choice Meaning I violated the first I evaluated the second as well because Of course the number is not a valid choice So this is the overview of the constraint system Another interesting subject Since we have such a complex data hierarchy Is how do we query for stuff like we want to find out what pages were created by Tim Whose french translation as a second paragraph body with the contents So who wants to try writing that as an sql query? I certainly do not It'll rip I'll rip my hair out if I have to do that The two pole has a much better solution. It's called a dot notation, which allows you to query into fields How many of you have used entity query as a system? Are you all pretty familiar with it? So I'm just gonna give a quick demo Another interesting thing about entity queries is that you can also query configuration entities, which means it's back-end agnostic I'm going to Figure that up here. I know this isn't valid, right? Let's try to make this valid What does that require? I need to say The value get value things I I could never get it correctly the first time It's a big design mistake in my opinion, but you know, if you keep trying you'll figure out where you are in the higher So now my node should be valid. I still have a validation I'll just crash my node do this. I should have an invalid title Meaning I can finally save my node. Okay, so now I have a node in the storage So buy it my storage Ask for all my nodes To tell me one result and if I put a condition on that I have no results on my title. I get one of my results back An interesting thing about that notation is I can query across Entities So my node is still here I will get it's okay. It's an entity reference field item. It's inspect. What's in it? Target idea zero, which means the anonymous If I set it to be admin target ideas one Let's say this is still getting my result, but I will add another condition. Who knows what the user name field Is on the user object that name? Ah What should I work unless uid is not the actual property. We do something fancy So one of the interesting things about entity queries is you can also use it to query configurable entities field configurations, right, which means If I want to find out in the system fields that use the text with summary field time field type. Well, this is an instance of using queries to examine what the entities are in my system We can come back later to Using dot notation, but there's other subjects. Okay, so more advanced topic now the data definition How do we create all of these fancy objects? Well, there's an intermediate system we have to go through which is the data definition object Which is sort of like a builder pattern but not quite Which means on Every type of field we have we have to define we have to set what the field definitions are and that's A list of data definition interface objects keyed by the name of the property so for example In the body field We have the value we have the format and we have another special one called process and the actual field object Needs to tell the builder system. Okay This is the property of the value This is the property of the format and this is another special particle process So every field class as at the end of that are data type plugins So when we are looking at our text filter This is The actual object that wraps the filter that gives us our possible values It's annotated with an identifier for that data type the filter format This is also where we define the constraints for validation My other time I thought it was like an hour Very sorry I'm just going to very quickly create an object And skip serialization normalization So to create objects you call up the type data manager identifier of The definition that I want to create either create elementary data types or I can create Field items that are more complex and this is the definitions the metadata will allow me to create my field Once I have this definition Then I can create the actual object But first You can expect that a little bit Okay, this consists only of metadata about what I'm about to build So if we look at The constraints for instance on a body There are none but If I do an integer Get the constraints Then I get the permitted type constraint again After that when I want to create the actual data object I have to use this data definition object And again using the type data manager Just want to see what the problem interface is first It's a Create here that takes the data definition object. Yeah, this one here and then you pass it the value That Should be able to create an empty Body field here we go the text with some of you This is the kind of magic that allows other systems instead of database starts to work I'm going to very quickly Go to serialization Right Which means Yes, or I can just get its normalized represented with this data I couldn't manipulate this data or read it from a API But the magic happens when I try to turn it back Into my object the express version of serialization, but I had more time last year. We're not gonna do fancy stuff Thank you for coming. You've been great. I hope I help you out You still have time to go to the bathroom