 Okay. Great. Well, welcome. My name is Joe Purcell. We'll be talking about unit testing today. I am an engineer at Palantir. I've been part of the web development community since 2003 and I love learning and teaching. So this presentation is actually inspired from my own experiences with unit testing. So I am excited to share what I've learned. Who are you? I hope most of you know something about unit testing and that you also want to get better at unit testing. If I were to ask you, multiply these numbers together, you would have some algorithm to find the solution. So maybe you write it on paper. This is actually a competition and there are people who can do 10 of these in their head in four and a half minutes. What if I were to ask you to write a bubble sort? Maybe you use some sort of test-driven process to write the code or maybe you're very familiar with bubble sorts and you do it off the top of your head. Now, what if I were to ask you to write Drupal 8? How would you do that? Well, you would have a combination of processes, I'm sure, but the point is we need processes. As humans, we need processes to do things well. So if we're doing arithmetic, algorithms will help us do that. Test-driven development, that's a practice that will help you write software. Solitary unit testing will help you write well-designed software. So that's what I want to talk about today is solitary unit testing and how that will help you write good software. So we'll start with the definition of what is solitary unit testing and we'll look at what test doubles are and we'll go through principles. We'll take a look at some code samples of solitary unit testing in practice and then we'll look at some test smells. So you've been using solitary unit testing for some time and you're running into recurring issues. We'll look at some common issues and then we'll conclude with some motivations. So why is solitary unit testing a good thing to do? Before I go into definition, how many here are familiar with unit testing? Okay, excellent. If I were to say mock the boundary of a class, how many would feel comfortable giving an answer to that? Okay, perfect. So we'll go through that. So definition, let's start with what is a unit test? Unfortunately, there are many definitions. George Mazaros and his ex-unit book has tried to consolidate these definitions. What I find most commonly quoted is Wikipedia, which is like a whole paragraph. Well, I will propose or I will point out one definition that I really like, which is Martin Fowler and he says it's a situational thing. So the team decides what makes sense to be a unit for the purpose of their understanding and the system under test. I like this because it gets at the heart of what we're after, but it's hard to build on a definition like this. So I propose that we define it this way. Unit test verifies a function's correctness against a specification. So a unit test is something that verifies that a function, which in this case, we're going to say that unit could be a function like T or it could be a method on a class like construct or get name or whatever. It could be a script as well. But the point is that a unit is one thing. A unit is just one thing. Well, when we're testing units, there's two challenges that we run into. There's indirect inputs, which we call shared state. So this might be global variables, might be static methods, etc. Or it could be data provided by a collaborator. And there's also indirect outputs. So indirect outputs would be side effects. So maybe you're writing to disk or maybe you're calling some method on another class and that has its own behavior. These things make it hard to test a single unit. Basically anything that makes a function not a pure function is what makes unit testing hard. Or at least that's one of the things that does. Well, test doubles. Test doubles are the tools we use to deal with those side effects and shared state. So how many of you have worked with test doubles know what test doubles are? Okay, great, about half of you. So I'll go, there's like six of these. We'll start with dummy. A dummy is just, it's just an object or a null value that doesn't get called, but is needed for instantiating or using the class under test. Fake is very similar, except it is called. And it's a fake of the actual object. So an example might be you might have a fake file system or a fake database. A stub is you double the object, but it is called and you can figure what values it returns. So maybe you have some data provider and it returns an object for you. A spy is similar in that it can provide indirect inputs, but it captures outputs. It captures indirect outputs so that you can verify that certain methods were called after the fact, after you've executed the class under test. And finally a mock. And we'll go through each one of these with some examples. But here's the overview. So and then a mock, you can also provide indirect inputs with a mock, but it does a little, it's different than a spy in that when it verifies indirect outputs, it does so at runtime, or when the code is executed. And you can figure the mock to run those verifications. So we'll see what that looks like here in a second. So a dummy, here's an example of a dummy. So let's say you have some class and this is, so these examples are using PHP unit. So hopefully you're familiar with PHP unit. So this example here, the dummy variable is going to be instance of some class. And let's say in this case, the class under test, it needs, you can't just use a null value here for the dummy because of typecasting, let's say. So right here, we're just, the method doesn't need dummy method is, let's just theorize and say that some method doesn't need it. So this would be an example of using a dummy for your test. If it is called, that would be one of the other five. A fake, it's very similar except it's not, the fake actually has logic. So let's say in this scenario, we have a fake file system in our class under test uses the file system. So let's say, maybe class under test is a logger, let's say. Maybe the fake file system, whenever you write to disk, it just returns true, you know, a success value. That would be an example of a fake. So we're not using the real class. We've implemented our own class called fake file system that gives us the same behavior. And a stub, a stub, the syntax is similar to a dummy except we're specifying what each method will return. So in this case, we're, and we probably wouldn't do this because user is a value object in this case, but I'm just demonstrating what's going on here. We're specifying that the first name will return first and last name will return last. And then we assert that the get full name will return that exact string. So it's like a dummy except a stub will get called and you specify the return values. A spy, you can do the same thing with a spy. You can, this example doesn't show specifying return values. But in this case, we're just doing behavior verification. So previously, we saw state verification, we verified what the return value was. In this case, we just want to know that get first name and get last name was called in this made up example. So it's like a stub, but it's capturing the calls and we're verifying them after. One quick note about spies, PHP unit does not have support for spies. There's a hack you can do. But the recommended tool to use is prophecy. So what we're seeing here is an example using prophecy within PHP unit. And lastly, there's mocks. One quick note about mocks, a lot of people use the term mock when they may not actually be using a mock, they might be using a stub or maybe even it's just a dummy variable, dummy object. PHP unit does not help this at all. Because everything you have to call get mock. So anyway, so in this example, if you look at the second two lines, we're expecting that get first name was called once. And we're saying that it will return first. And then here we're saying get last will return last. And then we exercise the class at the last line saying get full name. So when this runs, it's our mock object itself is what's verifying it. So this differs from a spy, a spy, we have to run those verifications later. And the mock example, it runs as verifications on the mock object itself. So here's the recap. Does anyone have a question about any of these test levels? Does these make sense? More or less? Yes. Yeah, so, so your question is, is a fake a concrete class while the others are made on the fly? For PHP unit? Yes. I can't speak to other testing frameworks. But yes, a fake would be, you have a real concrete class, let's say it's a file system class, you create a fake version of that. That's also a concrete class. These others, the stub spy and mock you in PHP unit, you're going to call, you know, get mock, and that's going to be created on the fly for you. Anything else? Cool. Let's, let's continue. So here's the question, what should you double? So you know what doubles are, you're familiar with the unit testing, you've probably even used mocks before. What, how do you know what to double in your class under test? Well, it depends. If you're doing sociable or solitary, we're going to look at both briefly. So you're familiar with what the difference is. Sociable means that you cross some boundaries. We'll define boundary in a second, but basically a boundary is going to be your calling method on a collaborator class, maybe you're writing to disk or reading from a database, those are boundaries to your class under test. In solitary, you never cross those boundaries. You should never cross those boundaries under solitary testing. With, and here's the second thing is that with sociable, you can have more than one concrete class instantiated during, during the, the test run. With solitary, the only object or the only class that should be instantiated during the test is the class you're testing, with the exception of value objects. Value objects are okay, because they don't have behavior to them. This was coined by Jay Fields, working effectively with unit tests, highly recommend his book. So what is a boundary? Let's be a little more specific. William Caputo, I hope I'm saying his name right. He wrote a blog post on this and he says it this way, so a boundary is a database, a queue, another system could be like a network call to you, or even an ordinary class that is outside the area you're trying to work with or responsible for. So basically it's a indirect input or indirect output as a boundary. What does one concrete class mean? What that looks like in practice, it means that you're going to be doing things like using a double for dependencies. So if you have a class constructor and you're passing in some dependency class, you're going to use a test double for that class that you're passing into the constructor. It also means that those collaborators, if they return objects that are then used by the units that you're testing, those should also return doubles. It also means that statics would violate the principle of solitary unit testing. And again, there's the caveat of value objects are okay. They don't have behavior. If they do have behavior, then they would have to follow these other principles. And yeah. So as a whole, here's how to think about solitary testing. Think about it this way. So with solitary, you would use unit test base. With sociable, it might be a kernel test. Symphony has kernel test class that you can implement as well. And then there's functional. So when you think of the lines of code that would be executed during test, the functional is going to have the it's going to execute the most lines of code. Solitary should be the fewest, but solitary gives you the most control. One side note. You can do all of these types of testing with PHP unit. So don't be confused. There are still different strategies. They're different approaches to testing. And as you test, you should keep these in mind to use the right type of testing strategy. So principles. So let's move on to looking at some code examples. The examples I'm going to use, we're going to follow the format, the triple A format, which is arrange, act, and assert. So you're going to see first lines of code are going to be arranging, we're going to set up our mocks or spies, set up the class we're going to test, then we're going to act on that class. So we're going to call the test method, we're going to call the unit that we're going to test. And then last, our assertion should be the very last thing in our test method. So visually you should see this distinction. So the first example I'm going to look at is batch storage. There's a method on batch storage that looks like this. There's a number of things going on here. I'm going to say that I want to test that the session is started when I call load. So that's what I want to test. So this example I'm going to use is a mock. So I apologize for the highlighting. I couldn't figure out how to, with reveal, point out this one line. This one line I want to look at is, it's in bold and it says session expects this one's method start. So that's our assertion for this, for this test. And when I run it, when I load one, do three, this is going to execute. And if start is called, it will succeed. But there's a problem here. We don't assert last. This is why spies are useful. So let's try an example with a spy. In this example, we are able to assert last. So one little caveat when you're using mocks, keep in mind that you are creating your assertions or expectations at the beginning of your test method. And over time, that can be very confusing. If you have hundreds of tests and you're reading these tests, it can be very confusing because you have, well it depends on if you're using an object mother or a data builder. You might have fewer lines of code. But in general, assert last in your test methods. Second example. So this one's a fun one. So this is the maintenance mode subscriber. So this does two things. This class will, if the route matches and the user is not able to see this page, it'll return the maintenance page. Otherwise, it'll tell the user some message. There's a problem with this. We can't solitary test that. And here's why. There's a static call. There's actually three static calls. And the function called the Drupal maintenance theme loads a lot of things that I don't want to deal with. So what I'm going to do is I'm going to delete the untestable code and reimplement later, which we have to. We still need to maintain the same functionality. The second thing I'm going to do is I'm going to break this subscriber into two subscribers. There's going to be one for setting the response. So this should, the entire purpose of this subscriber should be if the request matches return the response. The second one is going to be dedicated to showing a message. So here's our revised class. We're going to call it maintenance mode page override subscriber. For this class, though, I've created a maintenance mode page interface. This is going to get the response. So this is going to allow me to simplify the class. So you'll see this in play. This is what the maintenance mode override subscriber looks like now. So all this does is it does the same checks as before and then it gets the, it just calls maintenance page, get response. What's really nice about this is I could have someone come and help me out and implement a concrete class for a maintenance page because we can test it without a concrete class. What's nice about this is our class now does one thing, which means that we can test one thing and it now has three dependencies instead of six. It's more readable and it's easier to test. So this is what the test looks like. The test before, which I've written, is it's quite extensive and it has a lot of caveats to deal with the two cases. But this is a simplified version. There are a couple things I want to point out. So as mentioned, the maintenance mode page interface, we're not using a concrete class here and we don't have to because we're using an interface. So I could have someone else help me out and they could be working on the maintenance mode, a concrete version of this that is actually used by the application at runtime. The second thing I want to point out is that we are asserting, we're making sure that the get response on the maintenance page is going to turn this fabricated response, which we are going to use at this last line here. So this test is verifying this one thing, which is if the request matches, I get a response set. That's all it does. So when you're going through your when you're going through and you're doing unit testing and you're following the principles of solitary unit testing, you run into problems, you run into challenges. And here's a couple that I've, I wanted to identify. One is obscure test. So this is a scenario where you're looking at the test, maybe someone else has written this test and you take a look at it and you have no idea what's going on. This is very easy to do. I mean, you saw these couple examples here. I don't show those test examples as good examples. I'm just showing the principles of here's how, you know, when you modify code, it's easier to test. You can make your tests much simpler and there's a couple ways to do that. One is to use an object mother. So an object mother, its purpose is to return mock objects or it could be value objects. So this user mother, for example, would have a method called get John Doe and a return someone named John Doe. If you have some other class, it would be, you would follow the same principle. You just have a bunch of methods for different types of mock objects related to the topic and return those. A variation of this is called a data builder. Data builder is a little more expressive. So I might call, you know, user builder with no posts and build and that's going to return a user with no posts. Or maybe I want one with 10 posts. Using object mother or data builder, they're going to reduce the number of lines you have in your range. So big part of obscure test is your arrange step has lots of lines of code. If you have that, try an object mother, try a data builder. They can help you be more expressive and concise in your tests. Another one is a custom assertion. So maybe you have a complicated class that you're wanting to test, something very specific. You can write a custom assertion. So don't be afraid to, you don't have to use only PHP units assertions. Maybe you have, in this case I show, I have a trait called user assertions. And on that trait there's a method called assert has no posts and there's actually a typo here. I would need to pass in the user to the method, right, because there's no other way I would know. But the idea is that don't let yourself to just PHP units assertions. You can write your own and you can be more expressive in your test method in your assert step. So if you're writing lots of tests and they're obscure, you don't understand them because of the assertions that are being made, try this. Try writing some custom assertions. Another one is, and I see this a lot, is when you don't expect literals. Expecting variables is sometimes okay, but in cases where you have 100 line tests or in cases where you're reviewing lots of code, it's really nice to see exactly the string literal. This is a very simplified example, but there are cases where you in your head can't figure out what the something variable is. So if you have, so some of these smells can combine. So if you have like assertion roulette or you're having a difficult time figuring out what exactly is causing the problem, having expected literals will make your test more clear. Assertion roulette, which I just mentioned. So this is a case where you have a test that's failed and you have no idea why it's failed. This happens a lot if you have, if you use a lot of mocks with a lot of expect statements, it's easy to get into a situation where your tests are failing and it's not clear why. So recommendation would be to try one assertion per test. So use spies, perhaps, and put this example here. I'm showing a bed example, but it's actually not that bad. I just wanted to show the idea of if you have multiple assertions, if you're asserting two different types of things, that's bad. In this case, we're just writing assertions on a value object, which isn't so bad. But anyway, two options you have. So one is make a test for a specific purpose. So break your test out into multiple tests. The other is write a custom assertion. That's another option to simplify. A fragile test. So maybe you have a team who's making changes and you change something and someone else's tests break. I was actually just overheard Joelle yesterday talking about how he was working with somebody and they kept changing things and someone else's tests would break. That's a fragile test. With functional tests, that's really easy because there's a lot of code under test. But what you can do to mitigate against that is have, eliminate your dependencies if possible. You can do that by adding abstractions. Eliminate side effects or shared state. Also avoid over specifications. So a common situation you can run into is when you want to test something on a class. But what you're actually testing is the behavior of another class. Sometimes this is necessary. So sometimes if you, common example is like a query builder. Well if the query builder, if you need to test a method that's using a query builder, you need to provide return values. So sometimes you can be overly specific about what you're returning with a query builder. So sometimes it's necessary, but if possible avoid it because it will cause you to have fragile tests. Another one you want into is test code duplication. So this is where you run in a situation where you realize that you're writing lots and lots of code that's the same. A lot of people want to put, if they're duplicating code, they'll want to put it in the setup method. It's not always the best. If you can put something in the setup method that's true for all possible test methods, that's fine. But more common than not, you're going to get better mileage out of using an object, mother, or data builder. And it also might be an indication that there's high afferent coupling. So with solitary testing, again you're going to be mocking the whole boundary of the class. So if your class uses a lot of other classes and you have a lot of test methods, you're going to have a lot of repetition of mocking the boundary of that class. Lastly, high maintenance costs. So if you run in a situation where you find yourself spending more time changing tests than code, you might be using the wrong kind of testing. Solitary unit testing doesn't solve. It's not best suited for all types of testing. In some cases you might want to do sociable testing. Those would be like kernel tests. It is fine to remove tests. If you have tests that aren't providing value and they're getting in the way, or even if they're not getting in the way, but they don't provide value to the code, consider removing them. Again, using an object mother or data builder will get you better mileage with your code because you're consolidating all of the arrangement. And adding abstractions where there's high coupling is good. And again, over specification of tests, if you've heard of cascade failures, that's what this is. You're running into a situation where you have multiple tests doing the same thing. The best advice is from Grandma Beck. This is a book called Refactoring by Martin Fowler and Kent Beck quotes his grandma saying if it stinks, change it. If you're running into situations where either your code is very hard to solitary test or situations where the tests you're writing are really hard to do, find a solution. But be careful. Solitary unit testing makes bad object-oriented programming stinky. So if you find yourself having a difficult time writing a test, consider whether or not the code you have is what's bad, not your testing strategy. So keep that in mind. Okay, so lastly, motivations. Solitary unit testing enables higher code coverage. Why? It enables it because you have control over all of the side effects and you have control over the global state or shared state. This allows you to configure which lines of code get run and you can be very specific with it. Solitary unit testing promotes decoupling. How does it do this? Well, it is very painful when you have closely coupled code. You find yourself repeating yourself. So solitary unit testing makes it very clear where the seams are in your code. If it's easy to test, it's going to be more likely than not. It's going to be decoupled. It's also faster than sociable. So if you have like Drupal, you have thousands of lines of code, having a solitary test suite is going to give you quick feedback. Solitary unit testing also enables better software designs. And this is the main point that I want to get at. I love this quote. It says, writing unit tests is reinventing functional programming and non-functional languages. It's maybe you've heard this before, but pure functions, functions that don't have any shared state, they don't have any side effects, they're easier to test because all you have to worry about are your inputs. And all you have to do is do state verification at the end. So this statement is sort of true. The thing is, solitary unit testing motivates us towards pure functions, but allows us to keep the benefits of object orientation. Simon Honeywell, he has a book on functional programming and PHP, which is really interesting. And he says this, it should not be underestimated how much easier these functions are to test than their state-filled counterparts. So where do we start? So if you are working with an existing code base, or you're starting with a new code base, there's a couple things to look for. I'm going to give an example of Drupal. So PHP metrics, have you guys heard of PHPmetrics.org? Couple? Okay. It's great. So basically, PHP metrics is a tool that does, gives you like Halstead metrics, gives you like 50 different metrics with some nice diagrams. It produces a HTML file that's really nice. So this is a graph that shows efferent coupling and cyclomatic complexity. I actually chose these two axes. The reason I chose these two axes is because what efferent coupling means is ones that are on the far right here, those are classes that know a lot about other classes. Those are going to be really hard to test. Why? Because if you have 49 classes that you need to write a mock for just to test a method, it's going to be very painful, right? So this is an indication of either A, you can't or don't want to break it up. Like these dependencies are necessary or it means that this code is highly coupled and you should decouple it to make it more extensible, more maintainable. So that's the x-axis. On the y-axis, cyclomatic complexity. The reason I like this is it represents the paths through your code. So if you have lots of paths through a single method, that means it's doing a lot of computation that one method and it's going to be really hard to test. It's going to be hard to test because if you have, you know, 500 different cases that are real cases for this method, you're going to have a crazy test method for that to write. So as you go up the y-axis, you should be thinking you do want to write solitary tests. You want to write lots of solitary tests. You don't want to write sociable. You want to write solitary because you want to get as many paths through that method executed as possible because it's very complex. So either that either have lots of solitary tests or B could be an indication that you're doing something sufficiently complex that you need to break it into multiple methods or maybe even just simplify the method. So these are each bubble represents a class in Drupal. I want to point out a couple. So on the top left we see archive tar. It's complicated. Tara's complicated. So in this case it may be something that it may not be worth, it may be more efficient overall to rewrite that class and start from scratch and start breaking out the complicated pieces into smaller pieces but you can use solitary testing to iterate towards that end. A couple others I want to mention. So entity manager, that might be a situation where depending on 49 classes might be very unmaintainable. So that might be an example where start with solitary testing and break it down into smaller classes. However, Drupal kernel, that's an example where maybe you can't practically break it down. Maybe those dependencies are necessary because it's Drupal, it's huge. Like the kernel is going to boot a lot of things. So in that case you would want to have lots of social tests, sociable tests, which we do. That's what the kernel test cases are for. So if you have an existing code base, start with data. Start looking at things that are high along either one of these axes and those are going to give you a lot of value from your test starting out. Why is symphony so successful for reuse? I think there are a number of reasons, but two things specifically that are related. It's good object-oriented code. If you've read symphony source code, it's very clean. It's very easy to read. And this is not a surprise. Most of the tests are solitary. Maybe entirely. I haven't read all the tests, but most of them are solitary tests. So let's recap. So solitary unit testing, you never cross boundaries and you want one concrete class during tests. And a boundary is an indirect input or output. And test doubles are the tools that you use to draw that boundary. So what does this mean? What does solitary testable code mean? It means that you have lower coupling. Because if you can solitary test it and maintain it, and it's easy to do, then it's going to be decoupled. Because those seams are really, if you have complex seams, complex interactions between classes, it's going to be really hard to do. So you have lower coupling. It means fewer merge conflicts. So if you have very clean interactions between classes, when you change to classes, decoupled code is going to have fewer merge conflicts. D8, we're looking to move to future branches. Having decoupled code is going to make managing those future branches much easier. It means higher collaboration. It means that we can divide up into teams and tackle different portions of the code, which means faster development. We don't have to have as many blocking tasks. It means quicker innovation. We can say we're fixing some old bugs while we're innovating on new things and extending the code base. It means that we're closer to Drupal 2020. If you listen to Larry Garfield's session on Drupal 2020, some of the highlights we're talking about, you know, moving to needing to do asynchronous computation, having thread safe code. Those are things that are going to be easier to work with if you have loosely coupled code. It means that you can have cake and eat it, too. So please come to the sprint on Friday. All skill sets are welcome. If you're a first time sprinter, there's going to be mentors. Feedback is welcome. There's lots of references and further reading, which I recommend. The main ones I recommend are highlights for me personally that if you were to ask me, hey, what would you recommend in order to have clean, loosely coupled code? Clean code by Robert Martin. He also has a great article on principles of object-oriented design, and Mike Bland also has a really good one. If you want to hear a contrary opinion to unit testing, I recommend, let's see, yeah, why most unit testing is a waste. That one is really interesting. I actually really like the points that are made in that article. And then two more I want to point out. The go-to fail, heart bleed, and unit testing culture, as well as how to prevent the next heart bleed, I think have very interesting suggestions about the future of the web and security and stability of this after writing. So thank you. The slides are available on GitHub, and I realize I'm a little bit early, but let's, yeah, any questions? I can also go through some more code samples if you want. Yes? We've always found that there's a difficult compromise between breaking tests up into simple unit tests, small tests, and the time to execute them all, and that actually maybe more with what you called, I can't remember, what's the next one up from unit tests, which involve other things? Like functional tests? The one in between, we call them integration, but the social tests, especially with those, we found quite often that it was useful to have a lot of asserts in the same test, because otherwise when you do the setup, that takes a long time, and the shorter the test run, the more likely developers were to run the tests and write them. Have you got any sort of helpful tips about how to manage that conflict? Yeah, that's a great question. There's no real hard and fast rule, but I would recommend looking at custom assertions, maybe a good way. I mean, it does depend on what you're asserting. I would say two things. One is, it could be that you would gain knowledge from a custom assertion, or it could be that you really are testing multiple things, and in that case, if you're testing all of it in one test method and it fails, it can be unclear which and unclear why it's failing. And really, this is in reference to if you have, again, like thousands and thousands of lines of code, hundreds of tests, it can be, it's really nice when it fails, and it shows you exactly why this test failed. Yeah, that's something to balance is the performance piece, which is why if you do, I'm curious, are you doing, so you were referring to the integration tests? Yeah, I guess one example I've got in my head is we had something which was testing the output of a, it's provided a kind of a spreadsheet output for financials, and it was really complicated to get everything set up to do the form the calculations that the class that provided the financial output was doing, and it seemed really good to me at the time to kind of have an assertion per value output, but all in the same test method, because at that point, you got, it was really clear which thing went wrong, because there's only one assertion for each value, but doing a different method for each value would have meant to set the whole data set up for each time. So was the main concern speed? Yeah, absolutely, yeah. Execution time, yeah. That's interesting. Well, so this code, it was, you feed it a spreadsheet, it does computations and returns a set of values. Yeah, essentially, I was getting data from various places and putting together, so yeah, we had the stubs for four different external systems that are supplying data, and then there's, it was assembling those to provide this two-dimensional array effectively of values. Yeah. Well, at the integration level, it's a little more loose. I would say that if you were doing like solitary testing, be very specific about the test methods. So one possibility would be to use a data provider and you could run through different test cases. So one of the things I've done, I do have an example right now, one of the things I've done is in a data provider, you provide not only the values used that you're going to pass to the class under test, but you also provide the expectations, like what the expected values should be. So you might be able to use a data provider to have one assertion where you're testing a specific thing and just run over each of those data providers. That way your test method is simplified, but I don't know where the data provider, if it rebuilds, it may run, I think it runs set up every time, so you probably had the performance. I think, I mean, probably, I think you answered this early on when you said that probably the code itself needed to be written in a better way for testing, but it was certainly something that, you know, I found it very hard to persuade my development team to write tests and to execute them when the test took 35 minutes to run, but when they took three minutes, it made a huge difference. Yeah, well, so with like integration tests, like having tests that take that long, you know, sometimes it happens, but yeah, I would say if you're not able to write solitary tests, I would recommend like re-looking at the code and seeing if you can divide it out. Yeah, yeah. We ran into the same performance problem on Drupal Core with our tests, and that's why a lot of the tests in Drupal are A, functional tests, and B, do an awful lot in a single test method. I found it's, unless you have actual solitary tests, it's often not worth pushing for one assertion per test method. It is worth pushing for one action per test method. So if you have one action that you then need to run for assertions on, yeah, whatever, but don't put more than one action into one test method. And if you have to do that for performance, it means you'd need to refactor your code so you can do solitary testing of the pieces and then just a single social test or a single functional test to make sure you've wired everything together correctly. But then the bulk of your work is done in solitary tests and then you don't have that expensive setup in that case. So that's where we've been trying to push Drupal Core emphasis on trying, but that's usually what I found is successful. Yeah, yeah, you should, that's a good point, you should have in your act step, you should only be calling one method. Yeah, the one assertion per test is a principle. It's not a strict rule. I just said, it avoids assertion and reliant, it avoids obscure tests. Make sure that you're, so danger with obscure tests is if you have multiple assertions that are doing, they're asserting different things. It can be confusing as to whether or not you've missed something, whether you've missed a test case. Because, yeah, because tests only, you only verify what you're looking for. There's, I mean, there's approaches to, like, yeah, there's other approaches to testing, but this here specifically, like unit testing, you're only, you can only verify what you're looking for. So obscure test confuses that. Anything else? Yeah, I mean, not directly related to the mocks and stuff, but what I've been asking myself, sometimes you have a component that creates maybe a markup. And you have, if you write a unit test for that, you have to tell the unit test what kind of markup you expect. And if that's like bigger markup, maybe, then it might be nice that the first one would just generate the markup and the next one would just make sure that it's going to be the same. And then you have some, if you intend to be, to have it be different, then you can run the thing that generates a new markup and then you make a good difference and see that, okay, that I'm happy with that. And from then on, your tests, compared to the new output. So is there some automatic way to have this kind of two way thing that in one way to run it is to generate this stuff? And the other one is to compare if it has changed? Yeah, so it sounds like you're talking about basically passing state between tests, more or less, right? Because you generate the HTML and then you want to manipulate it. And maybe, maybe I would have the HTML in a file somewhere. So and normally if I would run the test, it would compare if the output of some component is equal to this file, which is an HTML snippet or something. And but sometimes I intentionally want to change this HTML. And also when I run it for the first time, I don't want to write this HTML myself. I'd want to have it automatically generated. And then I can use a different mode to run this. And then it would just generate this HTML or update it. And the next time I use this usual testing mode and it would just compare if it's still giving the same output. Okay. Yeah, I don't think I follow. Yeah, I would say if you have a case where you need to, you want to, like you said, you're generating HTML and comparing it against a file. Yeah, for instance, I have one library where I generate HTML tables based on some input. And normally, if I would run a test and make sure that it's still the HTML output is still the same. And but sometimes I want to intentionally change maybe the HTML output. So then I would, the idea is that I don't write the intended output, the expected output, but that would be generated. I see. So, so the expectation that you're running, the assertion you're running is generated is what you're saying. Yeah, the expected outcome, the expected value is generated. Yeah, I would say at that point, yeah, that like that. Yeah, I imagine I couldn't build something. I was just wondering if there might be something existing. You can get false positives that way. So I would, I would recommend if, so indirect, indirect inputs, right? So I would change the code in such a way that you can control what you're passing the method is that's generating the HTML so that you can hard code, you can have an exact string of the expected output. And maybe we can discuss later. Yeah, I would like to say like I would be glad to talk in more depth. One thing about this too is like with this topic, it's very contextual. It's very case by case basis. So I would love to discuss. I'll be around through Friday. So to the last question, I've actually done something kind of like that, or I was generating markup. And what I usually ended up doing was just doing manual testing with it to make sure I got the output I was expecting visually and then to make sure I didn't break it, run it once, copy and paste the output into my test as a static string, and then verify against that. That way, I don't have to think about the white spacing and stuff like that. I can still do a straight string comparison, but I still get the test verification of, you know, if my output is changing, I know about it, which is what the test is for. So yeah, I don't have a tool either, but I found that just doing that manually is frequently good enough, as long as you're breaking things down to a small enough chunk. Yeah. Small enough task. Yeah. So testing serves a couple purposes. And in that case, like that's very important for regression testing. Yeah. Yeah. And just to clarify, I was just looking for a way to automate this copy paste step. So then it would be part of the testing code. Yeah. Yeah. I mean, if you know the, if you know the inputs and the output varies, then you have incomplete inputs. So yeah. I don't know. I would be curious to see the specific example. Yeah. Okay. Well, thank you.