 Hello Drupalcon My name is Alexey. I'm from Mipam company and today I want to talk to you about health The health of Drupal programmers. I hope there's such as this room Great, so how do you feel? good Are there any complaints? no Maybe it's because I just know you need test writers here They are probably busy with finishing their unit tests, but let's start anyway So a long long time ago in a Drupal version far far away There was a really happy time for programmers Every programmer could just write the code Without suffering with any of these unit tests And if someone suddenly came with a TDD flag and shouted let's write unit tests already They answered that this is not possible without Drupal because it has a lot of services which are impossible to mock and Here you can see a pretty famous quote of Angie Byron about this from far 2015 But since the first pioneer was Jonathan Wage in even more far 2008 for very old Drupal version 5. Do you remember it? He was inspired by this blog post about TDD with Drupal and He successfully failed at that time seems that was way too early But times change And now we already have almost 30,000 of unit tests in our Drupal core as you can see here And even several steps to simplify unit testing. I found only four functions here. They are But also I found more than 4,000 of active issues that are seriously stuck in the unit test status and they stuck there for years without any serious progress. I Also set out to see how things are going with that coverage in Drupal core and was very surprised to find only 15% coverage it's too little and One more find that made me think Both in Drupal core and in contributed modules I've noticed that they have much more functional test than unit tests even more than twice Since the tall developers prefer to write functional ones over unit tests for some reason So those developers why do they still not one or write unit tests and really why? Try to remember your feeling when you successfully Completed to writing the unit test after several hours of effort and it finally turned green Very happy probably yes, but satisfied. I don't think so It's because that unit test took you quite a few hours efforts and Produce it a huge number of code lines much more than the testing function And here is the usual picture when the actual code takes around five lines But the unit test requires to write more than 50 lines. Are you familiar with this? So let me try to do a little poll What kind of suffering from unit test do you have please write the hand who have business? Okay, maybe insomnia anxiety Maybe someone does not have any of the above symptoms Don't be shy write the hands Oh, I see the hand But are you sure that you're written exactly the unit test and exactly for Drupal? aha, good shot So I suffered all the symptoms in the past exactly like you but no more With my module test helpers. Would you want this too? Let's go. I'll show you the way Here is a simple Drupal function They just close three recent articles and builds a list of them nothing complex just 29 lines of code Here is what it does render just a list with three titles and dates and author names. That's it And now let's just try to unit test for it using the classical approach but probably not The most experience that unit test right there should have noticed already in that code the country factory entity query load multiple loading ends by references and even to link function all of these symptoms that writing the unit test will be a pretty tricky task and Most likely they will decide to write a kernel test instead. Let's do this too. Shall we? Here it is nothing serious very simple and short just 20 lines of code Execution time is five seconds not so bad But this is still not a unit test. It's a functional one and running a hundred such tests will take already almost 10 minutes in our pipeline that's not good and Well, we are here to talk about unit tests not kernel ones So let's see it brace ourselves and still write the unit test for it using the classical approach And how many lines of code do you expect in it? Let's get your opinions. Shall we? Here you can see a slide From an online pooling platform slider where you can simply vote using your mobile phone Just scan the core code or open a website and enter the displayed numbers if the core code is too far for you And the wait for the next slide Here it is so now just choose the option and While you are thinking about the right option try to recall how many lines usually you have in your unit tests This might get you closer to the correct result Also keep this slider page open and because we will have more polls on the next slides so don't close it and Those of you who already voted you can open the Q&A section in slider and start throwing in tricky questions for me We will discuss them at the end of the presentation And I see a lot of votes and most of you 96 lines and 72 lines also pretty popular Let's wait for a couple seconds to finalize And I see that 96 is the most popular one still Okay, let's finish it and The 96 line is the most popular option Ready to see what I heard for you and nobody was correct just because there was no correct answer Ended up with exactly 100 lines. Yes as many as the hundred lines just for our 29-linet function Just real times more lines Five times more than the kernel test nothing unusual because it's a unit test. It's doomed to be big But the execution time now is just great 70 milliseconds. It's 50 times faster than our kernel test also And we've got a hundred percent edge of the coverage pretty good But this hundred lines of code We've written as many as 43 lines of code just to mock the single n-stick query call and Perate lines of mocking for each node having 26 lines of mocking in total just for two nodes and setting up services for container builder took 23 lines and Suddenly we have all the hundred lines filled Are you satisfied with this? If you ask me absolutely not so Can you imagine a world where to mock an n-stick query you can write only a single line of code or Create a mock of entity with prefilled fields also the single line and Have a ready to use Drupal container with working core services out of the box What if I say that I have it for you do you take So I've rewritten the hundred line unit test almost from scratch, but now using the test helpers API How many lines of code do you expect now? Let's take another short pull Here it is. So you if you already closed the slide you can scan the core code again and just hold So obviously should be less than a hundred lines, but how much 72 48 24 maybe even 12 What I feel it And let's wait for more votes and I see that you're pretty optimistic and have 12 lines the most popular option Let's wait a little and Remember that you can already start typing some Questions, which we will discuss at the end of the presentation if you already voted And I see that 12 lines is the winner. Yeah So let's finish the poll with this and you are right. I Wended up with exactly 12 lines eight times less than the classical one And now it's very short and easy to understand isn't it and Look, it's half size less than the original function well lines versus 29 and even less than our 20 line at kernel test Let's take a closer look at the unit test code. Here it is The first line initialize the provided stop for Factory service and at the same time creates a new config record record with desired values This makes the highlighted part of the code works out of the box without any manual locks The second line initiates date for matter service Because we have in the tested function the conversion from time step to string that requires it and in the same line We are creating a medium format with desired pattern Next five lines Create the user Entity and for nodes with required values. Yes one line one entity not take line per each like in the classical approach and in this line of the magic happens The test help is module initialize the class with passing working stops for all required services and is gives our article list function It automatically provides the working entity type manager service with get storage and get various functions And it can even execute the entity query right in the unit test context It checks all query conditions and generate the correct result like in kernel test But in unit test and without any database It also provides fully working a lot multiple so you can just put the list of ideas and get the loaded entities Date for matter service that convert time stop to string using the previously configured pattern loading entities by reference and providing the label text also works out of the box and even to link produce a correct links to the entity and The final pretty boring stage when we are certain the results It's the same as the classic test nothing to reduce here Just checking is that the results Equals expected once and now let's watch the test shall we and We've got a hundred percent of the coverage. Well Now let's take a quick look to another approach here is the current test that executes the query Automatically we are a single magic line But that magic with the entity query works out of the box for now only for simple queries And for more complex queries with many condition groups. I have another solution Asserts in the crucial conditions and mocking the results manually Here it is This approach works well for any complex query you have even for database queries And I'll explain how to use it a little bit later. Now. Let's launch the second test and Now we've achieved 200 Centage of the coverage a hundred from the previous test plus a hundred from this one. So now we've covered twice. Awesome And since that's all from my side, but since not I Saw a couple of unit test writers recently Even after applying entity stops to pain points. I could still see that they continue suffering Looks like they need much more help and I have it Here is a brief list of pain points that the scalpers covers for now and let me showcase them one by one This is a group of archers to initialize Drupal services entity storages and safe entities They provide fully working stops of the real services and entities that can work well in a unit test context without initialized Drupal kernel Also entities are stored just in memory including configuration entities So no file is always called a by database is required Perhaps you want to know what this stuff is and The stop is a fully working class or service adapted to work in the unit test context without Drupal kernel and without database Here you can see a class that provides a stop for config factory service So it just extends the original class from Drupal core Initialize all dependencies set default for values and provide some helper functions Let's look how it can be used in real tests Here is an example of a simple and subscriber for config safe and delete events It just shows a message about this event. That's it. And Here is the test for the subscriber Those two lines initialize the stop for messenger and config factory services. After this we can use them like with a full Drupal kernel And here we have already two magic lines that do all the work. The first line is for save event and the second for delete event Under the hood the helpers checks that the service is really subscribed to the event Has the event subscriber tag and then executes the assigned function and this is done automatically without any code from your site The next group of functions are related to private properties and methods Using them you can easily get to set a private or protected property get and call private methods But some of you may have heard that using unit testing private methods is considered bad practice That's right But not with our Drupal We have a lot of abstract protected functions that we should implement in our custom code Here you can see a list of some functions and actually there are much more of them And they should be covered in unit tests. I'll show you how test helpers helps you with this Here is a protected function compute value that contains just three lines of business logic And we need to call it to check if they work right But with the classical approach we should initialize the full instance of the field with a lot of manual logs just to cover these three lines And with test helpers the things are simpler again Just a single mock and a single assertion And two magic lines that stood Also test helpers already provides fully working steps for many popular Drupal services like database, date formatter, modular handler and so on You can find the full list of services already covered by steps in the test helpers PHP file On the left side there are custom steps created over the original service classes And on the right side the list of Drupal services that can be initialized in unit tests context without any additional changes And if any other core services that not discovered yet Test helpers provides an easy way to initiate a full mock of the service as you can see in this example Let's take the renderer service that is not covered by step for now in test helpers And using the method test helpers service you can easily get a mock of it and set the desired return values for required methods The first example is a classical approach with method will return back You'll probably know And the second is already a test helpers feature that in addition to PHP Units way Allows you to bind a callback function directly to the service class So you can easily get access to this variable and use all private properties and methods In the renderer service a placeholders function is protected But you can just call it in your custom callback function That's very convenient And you can even initialize your very custom service using the unit service method It will automatically detect the location of your services YAML file in your module Find the related class and initialize it This group of functions is related to entity queries and database queries Using them you can check for crucial conditions in the query and set the results of the execute function And it simplifies the work even for all SQL queries Here is an example of usage that you've already seen before But you can say that we already can do this using the with consecutive function from PHP Units Yes, this approach will really work But until the moment when the order of condition changes But with test helpers the order doesn't matter because in real entity queries and SQL queries it doesn't matter too, right? And even if you add new condition the test will pass too Because by default it uses not strict mode and check only for these conditions And if you need you can enable the strict mode by the only listed flag And without it as you can see we've added the promote equals one condition And we still have a green test because it tests only listed conditions Not at all And without you can add them with any order And now I guess you are ready for a more complex example Let's get the cut-off service and throw into it some nodes Taxonomy with vocabularies and turns several users and to make it really painful of course translations and entity reference Ready? Here it is The actual code looks not so scary but believe me the scary will be when you will start to write a unit test for it I didn't even try to write it without my test helpers So here is the test that actively use the test helpers API Yes it's not as short as the first example we have already 41 lines here But it's still much less than we should write in a classical approach The classical one can easily take 200 lines or even more Let's elaborate on this code The first line initialize the language manager service and adds two languages French and German The second pretty large block is for creating two users at taxonomy vocabulary with two terms and two nodes But now for each node we have not a single line as I promised Well it's because here we are creating not a simple node A node with two custom fields and with translations I'll explain how it works Here is a custom field field synopsis It's just a stink field so we should only indicate that it should be translatable After this line the field can store translated values so we just put the translatable true and that's it And another interesting place is the field category It's not a base field so we should manually indicate that the type of this field is entity reference And the target type is taxonomy term To do this we just put the array with parameters and that's it After this the highlighted lines in the test function will start working as in kernel test So we can just call node field category entity to get the category And via the category reference we can get the label of the category Like in kernel test without any additional code And the next again one single line that does all the work for us After this magic we need just to call the method to get translation articles list with different languages and assert the results The first call is with entity with English language and it should return to values because both nodes have English translation The next two calls with French and German language should return only the second node because only it has translations to that language So the first node have only English translation and the second node have additional two translations And that's it, very simple as usual But perhaps someone is worried that with this approach it will not be possible to use the classical methods with mocking any method you want But don't worry, I'll keep this ability Via mock methods parameter you can pass the list of functions which you want to mock manually using the classical approach And even add new methods that are missing in the interface for some reason like send as email as displayed here And you can even pass a mock of any field to the entity staff So just use the create mock method to create a mock of a field and pass it to the field as a value That's it And if you have even more customised field with a very custom definition You can pass the mock of the base field definition like this And it will be applied not only to the first entity but for all other entities of this type and bundle too So just describe it the first call and all other entities will have this information automatically As you can see here we are applying the custom definition to entity one but getting it from entity two and this just works So I've showcased it most popular cases which this helpers covers for now But I don't want to stop and will continue improving the module I started the development of this module in July 22 related to the first alpha version in September After that I widely tested the module on several projects Gradually added new features with releasing several alpha and beta versions All other members of our team started to actively use it too and was very happy with this The amount of time spent on the unit test has been significantly reduced by more than five times So the management was satisfied even more than developers And for now I've released the first related candidate in April 23 And plan to finally release a stable version in the next month maybe And this model seems to be already used not only by us with our couple of projects But some other people started to use it as Drupal or statistic shows And yes the chart is going down in last month But it's not a negative trend, it's a feature I know that no one likes to install new modules especially only for unit testing So no one wants to write something like this in the YAML file of the module And show the dependency on some LXA But actually you don't need to do this, I've invented a workaround for this My module will not come to your production environment at all Because adding a development requirement is enough So you just use composite required def, Drupal test has 100% that's it And you don't even need to enable this module in Drupal Just having module file is sufficient That is why the usage chart in Drupal or goes down Because it can't only enable modules You can see an example of this approach in my another module, OpenTelemetry That already now uses test helpers to cover its logic by unit tests And one more tricky question Do we need to cover unit test functions by unit test? And my answer, sure, and I've covered them Not 100% yet, but when I have time I plan to improve this And yes, I've used test helpers to test test helpers And now, while not everyone is asleep yet, let's throw in a final poll Which Drupal helper you would like to have next in test helpers? Please type the name of Drupal service or file function With which you usually suffer the most in your unit tests You can write the renderer file system, permission handler Or any other service, the pane of which is still in your mind So just type something that you can remember with last unit test Or maybe even permissions Yeah, I'm already thinking about permissions handler and renderer tool For events, I've already implemented the helper So, since permissions is the most wanted one and OK, I will prioritize it The renderer plugins Thank you for your feedback, I will take a note about them And I think permissions is the most interesting one because Renderer is a pretty complex service But maybe for renderer I can invent something too So let's move on And you can add more results here And the results will save it, so I will analyze them and add something later So permissions is the most wanted one as I see Let's move on A couple of words about the roadmap I have no strict plan yet, just ideas what we could add and improve The first item is about adding new steps for more services And I already got feedback from you and I'm continuously doing this When a new service come to me, I'm trying to create a universal step Instead of mocking only for the one current case And the next item is about integrating with more feature-rich proficiency and macro libraries In addition to PHP unit Those libraries can significantly simplify some things But for now, I don't want to force users to stick with one of those libraries So just use it only raw PHP unit API for now So maybe in future I've not yet decided about this And maybe with your help I had something for PHP for proficiency and mockery users And together with unit test we also have kernel tests And yes, they are much simpler to write And I think Drupal already provide enough API to cover all cases But maybe in future I will find something to do in this area And the last but the most tricky one is about moving features to Drupal core Nobody likes install additional modules, especially if they are just for testing So I would be glad to have some features from my modules To push it into Drupal core functionality To make life of unit testers writers happier Because no one wants to install additional dependencies And having it out of the box will be pretty convenient So let's see what happens with this area And what ideas do you have? Don't be shy, just submit feature requests and we will discuss them in the tracker So now are you ready to use test helpers in your projects? Then please give a star and subscribe to new releases Or even better, join the development Because together we can make the module more universal and more useful And now that's really all from my side Thank you for your attention and don't forget the appeal Drawing their development Let's do the making life of unit test writers easier together But probably you have a lot of questions, right? Remember that you could add them right during the presentation Let's look what we have Since we have only one question for now What was the example module that uses test helpers? This is an open telemetry module that is available on Drupal.org I can scroll to this slide This one, Drupal.org project open telemetry This module provides open telemetry library available for Drupal So you can provide traces Especially it's useful for decoupled projects When you have something like react on the front end And Drupal on the back end And with this module you can see a full trace including Drupal Even Drupal is called queries And also my module includes all examples from the presentation So you can just download the module and execute all tests locally And debug them and see how they work And I see more questions Where can we find these slides? I will share this presentation Maybe I can put the link to this description of my test helpers module To make it available for all people Do core maintenance accept tests written with test helpers? I don't know Because now I use it only for our custom modules and our projects But if they just work pretty stable and convenient I see no reason to restrict core maintenance to accept them So now you can use them for your custom projects For your modules like my open telemetry module And we also can replace even kernel test to unit test with this approach Especially the test that requires some IPI for working with entities They can simply replace it What the questions do we have? How we generate those reports that show the coverage Just run PHP unit with coverage plaque and render the XML file There are several options You can't remember the exact one But it can generate the HTML report and the XML report and even text report So you can use this approach for your own tasks And since that's all Questions Let me check, maybe there are more questions What about using AI to generate unit tests? Maybe it will work But for tests even humans can't find a good approach But artificial intelligence, I don't know what it can invent new Because unit testing, AI logic should be covered by unit tests by humans Maybe you can try What else we have here And since that's it For now And also you can ask directly now And the host of this session will write me in the meeting chat about this But since that's all So thank you for your attention And if you still have some questions you can reach me by email I can jump to my contacts Yeah So you can write me by email, by direct message using Drupal Orc Or maybe in Slack And you can just start trying to use my module in a single unit test And it's not forcing you to use it in all unit tests Just start using it with one unit test And if it will work for you, you start using it in all the next unit tests And thank you again for your attention Let's see Thank you I'm glad to see the audience because the remote experience is not offline But I'm glad that I have this opportunity because I had a problem with Visa And online is a very good opportunity to get some little amount of Drupal Con And thank you Goodbye