 24 so everything gets kind of scrunched down so there'll be some kind of flickering back and forth. So make sure we got everything set up over there, I know it's kind of small, I hope everyone can see this. So this is the basic test we're going to go over. We're using PHP unit and PHP unit has a plugin for Selenium and for those who aren't familiar with it, I'm just going to quickly go through this. So I'm extending a web driver test case which is just a general object pattern of inheritance and this then obviously then extends a Drupal Selenium test case which is just another general object, kind of a helper object that I made and then this, I can open that up. This then extends PHP unit extension Selenium test case. And this is PHP unit's plugin that does the Selenium testing for you. And like I said, since it's open source, we can go and just kind of give anyone who's not familiar with it an idea, so I'll just open it this way. And inside of this PHP, you can see it's PHP unit Selenium and then inside of this class is function to command and all I want to show you here is that it takes these commands that we're sending it. For example, open, wait for element present, takes command, sends it to the Selenium server that's waiting at that end point and just uses good old curl since the ACP command gets information back and that's the workflow that we're working with. So I'm just going to run a simple workflow here or a simple test and the first thing I'm going to do is I'm going to spin up the Selenium server itself on my local machine. So you can see I have this jar file here, Selenium server, which you just download from the Selenium site. I'm just going to run it in standalone and just run it like a simple jar file. You'll see it's launched Selenium server and if I go to this port, you'll see it started on port 444. You can see that it's running and don't worry about the errors because we didn't send the right information. And then I'm just going to run this simple login test. I put some sleeps in because it goes really fast, so I tried to slow it down a little bit and actually apologize for that. So these two lines are a way I switch between WebDriver and Selenium. So later in the talk, we'll go through that. But I want to run that test again to show one thing and this will help when we go to the difference between Selenium 1 and WebDriver. So you'll see we have this control window up here and in the bottom we actually have the actual test running. So plain and simple. And I think we all understand that. Are there any questions so far? We're all good? OK. All right. So we did our simple test. So now we kind of want to talk about WebDriver. So WebDriver is Selenium 2. If you want, we can go into the history. But it was a separate project. But WebDriver is Selenium 2. And one of the main things that's kind of important about it is that they want to drive it from the user's point of view. They don't want to drive it as a browser, so to speak. And as I mentioned before, they drive it by the browser's native API. So for example, Internet Explorer, if you were going to write something for Internet Explorer in the most efficient way, you'd probably fire up Visual Studio and probably program in C++ because its automation bindings are in C++. And Firefox has, I think, XP-com, which is its kind of plug-in device that's native to it. And that's what WebDriver does. It drives a browser by its native API. So for example, even though it's written in Java, Java has something called JNI, which is Java Native Interface, which allows you to run C++ almost as if you're natively working with it. And so that's the way they wrote WebDriver's architecture setup. It's also an actual standard. You can go and Google W3C WebDriver, and they're trying to make it a standard so that other things can interface with it. So I don't want to get too bogged down in the details of architecture. That one statement of they're trying to drive it from the user's point of view is kind of very important. And it kind of also changes the way you write tests, and it can also break your tests. So I have this simple example. I call it the fortune teller. And what it's going to show is how this concept in place, a Selenium 1 versus Selenium 2. And also some of the side effects of writing in Selenium 1, how you can actually have tests that pass that shouldn't pass. So let's go ahead and go through the fortune teller. And I'm going to go through it manually first just so we know how it's set up. OK, so it's really simple. And I'm almost out of power. I apologize, plug in. So I'm going to put in any name, favorite color. Let's go with red, predict my future. And it says, Jim Brown, you're OK. It's very simple. I wish fortune tellers were that nice, so. So our test, if we go look at our test, is pretty straightforward. We're going to open the web page. We're going to get a unique name, just a random name. We're going to put the name in. We're going to click the Next button like I did. Wait for the color text box to come up. And we're going to put in a color. I arbitrarily chose blue. And then we're going to click Next. And we're going to look for that link that says Start Over. Now, just a little background about the fortune teller. I use the basic Drupal AHA framework for Ajax forms. So can anyone guess, I guess you can't what will happen when I run this in Selenium 1? Because we did the user login test, which means you put in the information in the two text fields, and you told it to click the button. And we're doing the same thing here, which is what I just did manually. So let's go ahead and run it. I really wish I had a 10 by 24 screen. Yep, not small enough yet. Especially green, to an extent. So it did put in the name. And actually, to make this easier for you guys, I'm going to run this in single window mode. So one hint is that the, and you can see it's now just waiting for that element to come up, for the color question to come up. One hint is that the AHA framework actually doesn't fire it attaches this button using a mouse down event. And as we mentioned before, the Selenium 1 tries to JavaScript. So when you say click the button, it uses JavaScript and actually clicks a button. It fires the click event. And the problem is it only fires a click event. It doesn't actually do a click on the button, which would fire all the events that would happen in the button. The mouse down, mouse up, and the click. So the difference is, when I run this same test, switching to WebDriver mode, and all it does is extend a different, extends PHP unit Selenium 2 implementation instead of the Selenium 1 implementation. Let me see, it works perfectly fine. Because it's driving it from the native bindings, and it actually just does a mouse clicked here. And then that bubbles down and does all the events that happened on that button. So if you've ever done autocomplete fields, for example, the onBlur event, usually you have to do a fire event onBlur. Because when you lose focus, it actually doesn't fire the blur event. When you use WebDriver, it actually fires. When you lose focus and goes someplace else, it actually fires a blur event because it's actually driving it like a user instead of through JavaScript, where you're kind of only doing one item and not the full stack. And I have another example. But do you guys all understand that? And the other example real quick is that, so what I did was I put in some JavaScript that just disables the name field. And again, if we go back to our simple basic login test, and I made a test called basic typing, it's just going to type in those fields and do an absolute assertion, which will always be true totality, if you will. And if I do this in Selenium 1, yes. We'll see that even though it's disabled, it puts the username in there. And if I switch that, or if I just switch to WebDriver, so it's not even going to type it because to WebDriver, it says that field's not available for me to manipulate. So again, from the user's point of view, that one line just changes how you write your tests and how WebDriver goes about throwing errors and seeing what's available for it to do. So you get kind of a real feedback. And these are very basic, simple examples. But once you start getting kind of into more complicated items, you'll get hit with things like if you have a drop-down menu and you try and do a get text to see what's in there, Selenium 1 will give you the full UL, the full ordered list. That's OK. It's up to the group. I have to do this. We just need to know what time you'd like to go to break. I can have the hotel ready to go with their stuff at that time. OK. I'm almost to the next section of organization. So like five minutes? Five minutes from now. Yeah, five minutes. And yeah, I could really use some coffee anyway, so. Good. Thank you very much. I'm sorry if you're thinking of me. That's OK. Where was I? Oh, yes. Oh, yes. So is it just a question? No. And actually, that's my next slide. So I'm glad you asked. So that's kind of the difference from a user's point of view. And when you do get text with WebDriver, you'll only see what the user can see. You'll only see the MyAccount. You won't see their logout or the other drop-downs in there. So it really is kind of saying, and then you get the more complicated things, something's overlaying that the user couldn't click. It really does kind of go from what the user can do. So that said, I was excited about WebDriver, and it's nice, and my tests are slow, and isn't just going to fix it because it's new and fancy. And the one thing I found out is that WebDriver gives me stability. It's more human-like like we just went over. Oh, sorry. It's OK. It's just a 40 minutes. OK. Yeah, I think. I thought you won't lose time. Oh, yeah, that's fine. I think I'm only till 11.45, yeah? OK, so I'll write it down. Yeah, but it's only have an hour session. Yeah. Yeah. Yeah, that's OK. And there's less what I call JavaScript hacking. If you've worked with tests, I'm sure we've all done weird fire events or we've actually pulled the browser bot, which if someone's not familiar with the browser bot, it's the actual JavaScript from Slonium 1 that drives a browser and kind of gives you a truer picture. So that said, the one thing I didn't mention and the differences was speed. And even though it's using the native API's stability and the user's point of view was really what they were driving for. And what I found is there is not really a big difference in speed between the two. If anything, I think WebDriver might be a slightly slower, a tidbit here and there. But it's not a magic pill where if you're having slower tests, going to WebDriver 2 or going to WebDriver is just going to give you this mass or a massive 20% speed increase. From working with it and from doing some test implementations, I haven't found any difference. I shouldn't say any difference, but much difference between the two. If you're looking for speed, moving to WebDriver isn't going to solve that problem. So to answer your performance and speed question, no, for my personal experience. So from there, I guess we're not breaking. Should I just go on to organization of tests? What, no? All right. So that's pretty cool. We've got a new tool. It's fun to play. Oh, yes. It's a WebDriver? The drawbacks to WebDriver, writing tests can, it's nice from a developer's point of view to be able to do JavaScript hacks. Or for example, it is frustrating to like, you want to check to see if something's in this dropdown or in this element that maybe the user can't see. But for you, you just need to assert that it's there. But you have to kind of test it in some other way, something that the user can see. So it's a little frustrating in that regard. Secondly, it's nice that it's using the native API. So that also makes development slower. So if you're using Selenium, they kind of have a fast release cycle, especially with Firefox, seeming to release every month in updates. But it does make it slower because you have this translation going on. Another drawback to WebDriver, I think those are the two major ones for me that I wanted to mention was it's not a magic pill. In my experience, it has made testing sometimes a little frustrating or harder. And unfortunately, I'm not working on it. But if I was working on the driver implementations, it makes it a little bit slower. Because yes. Well, I guess that's a double. Can we get a brief summary of the question? Yes, yes. The question was someone new to the organization writing tests, and they were able to use the Firefox plug-in that allows you to basically record your test. It's what they call the HTML unit test. Yes, yes. And so it allows you to basically record your tests and then replay them. So instead of writing them like this, you're able to record your tests and replay them. And the question is, will WebDriver allow this because it can only be used in the hands of a programmer? And I think that's a complicated question. So for example, with the IDE, you can record the tests. But some tests are more complicated where you need a developer to write the test. You have to do it by hand. And also I would say, yes, you will always be able to. When I'm switching between the two in terms of Selenium 1 and Selenium 2, you can use an IDE and record your test and run it that way. So with that regard, yes. But you'll always, I think, need, and especially in a larger organization, or a more complicated site, a developer to write the test anyway. So I do agree. There will be an IDE. And they can use the IDE. But you'll always have these complicated tests that you'll need a developer for. And not only the testing, but the architecture of fixtures and tear down and set up. And when you start to get into a lot of tests, like when we go through organizing tests, it's if, for example, if you have a hundred tests and they're testing some widget, and then the UX comes in and they totally redo the CSS, and the DOM structure changes, usually the IDE uses XPath. And if the DOM changes, you have to then go back and redo all those recordings. Where if you have, well, we'll talk about this. But there's ways to mitigate those changes where they're not as painful. Where you can go into one place, update this area, and then go forward. So it's kind of a catch-22, depending on which specific area we're talking about. Did that answer your question? OK. Yes? That's kind of a related thing. It is, too. But I should, with the caveat, say that underneath, I do have a translation layer. And for this talk, since I only have an hour, I didn't want to go too deep into. And we can do that if we have time at the end, migrating from Slonium 1 to Slonium 2. But for example, there's a slight translation layer. But it is, the translation layer took me a couple hours to write. It's not complicated. It's just slightly different functions. And so it's like, if you call this function, I just overwrite it. And this is the one that you should call. I'm there, so you've got water. You've got zombie now, which is the idea of speed and testing how difficult it is to write tests, et cetera. I mean, when you pick a path of what you're writing the test for, are you pretty much locking yourself into, say, Slonium? Me personally, I would say no. So the problem, I shouldn't say the problem. So with Slonium, like this open and wait for element, those are Slonium commands. But like I said, there's a translation layer underneath. So if you were to abstract it into your own domain language or pseudo language, and then have your own translation layer under it, which some languages do do, they have an abstraction layer. And then you can use whatever implementation underneath you want to. Then I would say you're not really locked in. Because I could, if something's method for opening a page is get the page, then I would just override the open function, just general object-oriented programming, and just override it and say, instead of that, use this. So I would say, yeah, on one hand, you are, because you're used to wait for element and whatnot and use that one language. But on the other hand, if you really needed to, and this is really going to help you, then you could go that way. But then that's a question of, do you just redo it anyway? So I hope that answered your question. Yes? There is a test runner for the IDE test. In WebDriver, I'm actually not sure if there is. I'm pretty sure there is. And you could definitely, because it's just an HTML table. So from a programming standpoint, yes, you can, as far as one standard implementation. I'm pretty sure there is. I know there is in Java, so I'm pretty sure there is for WebDriver to a test runner for the IDE or the HTML ones. Okay, yeah, so we're going to go through organizing tests. So when you start using the tool, you'll start, excuse me, writing a lot of tests. As a caveat, this information is taken from Selenium's own website. They have some great information and resources there. It's also just, from my personal experience, doing tests both in Selenium under Java with JUnit and also PHP. So just some practices that I found out in consulting and also at my current job with the Economist. So the first thing we're going to go over are page objects. And page objects allow you to get this separation of test code and what's called, they say page objects, but I like to just call it an object because I think that better describes it. And the object is just some resource, like a user or a home page. Or kind of think of it as maybe some feature. And users is a good example. And I'm going to go through a user example. But they're definitely useful for objects that you use a lot of the time. So for page objects, we're going to look at two tests. And so we have our basic login test that we remember from previously. Opens the user, puts in username and password, and kits submit. And then I made this test PO login, which is page object. You'll see I have this new object here called user. And I just give it an instance of the PHP unit Selenium test class. And you can see the difference between the tests. Instead of me having to know in my test, and I shouldn't care, but in my test login that I have to type in the edit name and edit password. My edit name, my edit password, and what the username and password is, I have this user object. And I can tell the user object to do certain things and then do my assertions after it has done its work. So for example, a login, I shouldn't have to care about, OK, how do I do that? And usually you have to go find an example. I should be able to instantiate an object and tell it to do a login or some other complicated, not method, but workflow. So inside of this DC user object is we just have a constructor. And we have a register user function. And the register user function just, where does that go? Sorry. Create user function just creates a random name, goes to the register page, fills all the information, registers the user, and then returns the object. So from there, you have a user object. And the object knows, OK, this is my username. This is my password. I know the information about myself. What do you want to ask me? What do you want me to do now? And then with that, I tell it to log out. And then I tell it to log in. I just do a simple assertion. Now, you can see, this is a very basic example, but you can see if you're, and we've all been there, if your registration and login process, especially your registration process is pretty complicated and you need to do it all the time, you could one abstract into a function, register user. But with a page object, it's more abstract and more powerful because you don't just register. It knows how to do all the things that that feature should do. It knows how to register or send itself an email or check its email. And then you do your assertions off of that. Does that concept make sense? OK. I just want to make sure I'm explaining everything very well. So I can run through this example where time's kind of short, does anyone want to run through it? No, OK. So next, I want to go over the example I kind of talked about before, which is user interface mapping. So like we were talking about, I kind of mentioned before, if you've got all these tests and you can think down the road, someone's going to say we want to change this feature. We want to change how it looks or we need to put in a new element. And then you've got to go through all your tests that reference that location, whether via XPath or some other locator, because once the DOM structure changes to locate it, you have to change how you find it. And what UI mapping does is it allows you basically, from a simplistic point of view, it's just constants. It's just for the location strategy instead of text. So in this basic UI login, what I did was I extracted it out so we didn't have to go into the DC user. So if you wanted to kind of build upon all these practices, you would take this practice and then put it inside of your page object. So that way we now have one point of reference for location strategy. But you'll see this DC user UI login user submit. And in this class, you'll see these just constants. And so you can see edit name, edit pass, edit submit, and then the login user account link. And so when I run this, I use those instead of having to type out the text. So now, if someone wants to change the name of the login field, or maybe I use XPath, and then there's a new div around it, so I have to change my XPath location strategy, then I just do it in this one area. And for the 15 other tests that are using it, if they're using this UI login, I don't have to touch those. So we're kind of centralizing all the changes that we would need to make that are kind of your day to day thing. Like you're going to change the color of the website, add a new button, and just making it easier to move forward, and not have so much pain when things change. And the other thing I wanted to mention from the page object perspective is that the page object shouldn't, usually, don't want to do assertions inside of your object. You want to leave that to your test. You just want the object to do the services that it should, like register, log off, it's a home page, create yourself, make yourself the most current, not the current. And what I mean by that in practice is I wouldn't have my user object, and I wouldn't tell it to assert is logged in. Because then my assertion is now in my user object. The way to do it is that I do have in here, this is my account link present. And all it tells me is it present or not. And so my assertion test would be, it's a slight change, but it's kind of important when you're making a lot of objects is keeping the assertions out of the object and keeping them in your test. Obviously, in the real world, you do put some assertions in there. Like, am I on the right page? If I'm not on the right page, get on the right page. Or if you want a dependency, I couldn't do this because of X, Y, and Z. So the whole thing is not going to work. But in practice, you want to keep your assertions in your test and you want to keep them out of your object. Was there a question or no? OK, good. So hopefully that makes sense. So make sure we're doing OK on time. OK. And then next, I'm going to talk about test APIs. And I probably should rename this just about helper functions. Kind of to the point of a new developer coming in, your test should be super easy to write. You should have tons of helper functions. Just so when you write a test, you're kind of talking to yourself. It's like, OK, I want to open a page, put some text in the header, see if it's there. Not thinking, OK, what's the name of that element up there? How do I get enough words to put up there? It should just be a very simple conversation with yourself so that even if a new developer wants to read through it, it's very straightforward. They're not thinking about all the technical, or the technicalities of how do I do that. For example, a page object, you use those to help someone, our new developer, or anyone on the team say, OK, how do I create a user? How many registration fields do we have? They should just be able to say, I need a user. I need them to log in. That's two lines of code instead of having to research. And it's also time saved having to research. Do we have new fields? Do we have someone, and so forth? I say, keep the actual test of small one-liners. What I mean by that is not having to, I need to register a user. So you have a block of code that registers a user. Split that down into one with a user object that says, register a user, done. Next line, assert that he was registered, or that he got an email, or whatever. But it's nice to keep them in to one line, so it does then read like a story. Utility function of your friend. Obviously, you're going to wrap some calls to make things better. And then we use this module called TestData with Drupal. Is anyone familiar with it? OK, TestData is a way to help you get your testing data in an efficient manner. That's, I think, a good summary of it. The way it works. Oh, great. While I start that back up, do you guys just mind if I just go through the code? OK, because we're running a little short on time. So why am I looking in there? So this is kind of a key area of it. And TestData is just a framework to allow you to get a set of data, cache it, and then loop through it. So if you think about a site that's got a lot of blog posts, but those blog posts may have different states. So let's take a lot of blog posts. So I've got 100 tests, and each of those are testing something about a blog. Does it have the right header? Can I edit it? If I log in as an editor, can I edit it? A regular user shouldn't be able to. You've got all these iterations, and you always need a random blog post. So every time a test starts, and we'll get to this scale, let's say you've got 30 tests, each of them are hitting the database, all trying to pull a random blog publish, or a random blog entry. And if you ever use the RAND function with my SQL, it can be a little slow. So what TestData does is allows you to basically set different data sets. So for example, I have a random blog one. And then I give it a query, and I tell it what size I want. I think I have it set at 200 or 2,000. And the URL format that it's going to use to get to that ID. So you see the NID here, and then you see the URL format. And it's just going to replace whatever comes back from here with that. But this stays in cache. So this query, it runs it, and then it puts them all in cache. So when you come back and you say, I need a random blog entry, it gives you one from cache. It gives you the next one. It's got an internal pointer. So then you have a data set, and then you can just go through it from cache instead of ever having it hit your database. This helps to speed up your test. Depending on, usually, you run this on your same site that you're testing, obviously, because it needs access to the database. And you can loop through it through cache. And then obviously, like we said, we have different states. So one might need a random blog. One might need a random blog with comments. One might need one without comments. One might need one with comments disabled. You can think of all the different iterations that you'll need. And instead of having to all come up with this framework of, OK, how do I get that? Or you've got this one query sitting in your test area. It all sits in here. It all sits cached. And it helps your performance of your test. Does that framework make sense? And you can see it in action here where it does. It looks for everything that's implemented at a certain hook. And then it pulls the test data from that. And then it goes to the next one. So it's like, go to the URL formats are go to, and then go to next, and then whatever your test data set is. So I'd say, go to next random blog, go to next random blog without comments. And it's all coming from cache. There is currently not a D7 version of that. I don't know if there will be, unless the community writes it, but or unless we write it, which is maybe a possibility. But the basic premise of it is not very complicated. It wouldn't be a very hard, I shouldn't say not very complicated, but it's not something that would be very hard to rewrite in D7. No, all the examples I've given here are using D6. But these examples are supposed to be agnostic of any Drupal limitation. It's really about testing an application. So with test data, you could write this in a different language, Ruby or something. It's more the concepts that I want everyone to take away from, not so much the specific implementation of Drupal or anything. For example, organizing your test is something you can do with any testing framework. But yeah, the examples I'm using in terms of, the only thing that's D6 is the site and also the module I made as a helper, which I haven't really touched on. PHP unit is what we've mostly been in. And I believe I'm using 3.7, 3.1.7. But yeah, so they've been in D6. But yeah. Yes. Say that one more time. The feature files. Did you do the keyword when then? The feature files? Oh, so feature or fixture? Feature files, right? Is he asking? Yeah, yeah. The GERF case and the GERF style. So why did I choose PHP unit to make the test? Well, because I want to use for this talk with WebDriver and Selenium. And for something like B-Hat and what you're talking about, it's a different paradigm. To me anyway, it's something that is everyone familiar with B-Hat and how it works. So B-Hat and I believe what you're mentioning features, you actually write out text. You say, you know, given that a user has logged in, next line, check that they have a My Account link. And then it splits this out into tests. It takes actual natural language and splits it out into tests. And then you basically fill in those tests. But from there, then anytime someone says, given any user's reject, so given a user's logged in, that test is already written. And so you get all these little tests and people can just write paragraphs or tests in natural English. Very easy. Am I following? Yep. I'm sorry? Oh, I haven't used it. Yeah, so Java does have it. But we're using PHP or we're using PHP unit just because in our organization, we don't have a lot of Java developers. But Java usually always has a better implementation, even in Selenium 1, than the other bindings. The web driver actually uses what's called the JSON wire protocol. That's what PHP unit uses. So it doesn't have the close relationship with Selenium that Java does. And I believe .NET has a close relationship also in terms of its implementation. So PHP's implementation is using the JSON wire protocol. And only those things are available to it unless we go use something like Behat, which kind of implements that. But yeah, you're right. And I think that'll always be like that, because I think, obviously, their Selenium is in Java. And they always take the interface. And I was frustrating in Selenium 1 because you've got this interface in Java. You can go read the Java docs. You're like, oh, that's the function I need. And you go and look for it on their interface of the protocol. And it's like, ah, it's not available there, though. Did I answer your question there? Yeah. Yes. Yeah. Yeah, I can actually. Yeah, we run a full headless test system. And all the tests work with Jenkins. We have a continuous integration environment in the last step of it. It runs a simple test. And then it runs all of our Selenium tests. Yeah, so I wanted to go through, actually, I wanted to go through an example of scaling out your development environment. But I was not going to go through setting up a continuous integration environment, though. That's kind of beyond, I mean, it's kind of beyond this. But yeah, I did want to go through scaling, or just at least talk about it. I don't think we're going to have time for it. But no, I wasn't going to go through a CI today. Just as a side note, there was a mention of Behat, which will be the next session on this track after lunch. And there will also be talks about CI and Drupal, CI processes and things like that. So just stay tuned. And see. So last but not least, OK. OK, we have a PowerPoint there. Apologize about Microsoft. That's true. Could someone call them over? OK, well, while that's loading up, I know what I'm talking about, so, or next anyway. So in terms of scaling, I think of it in three steps. Scaling your tests, and then scaling your dev environment. Oh, wait, hold on a second. I can't click anywhere. Yeah, because I'm on a 1024. Oh my goodness. Oh yeah, that always works. I can't even, uh, you gave me hope, though. That always works. I'm going to go, I've got a fix for this. So I need to start it now, because I'm looking at the PDF. All right, so the first thing I wanted to talk about was scaling the test. And is anyone here familiar with Paratest? So Paratest is a multi-threaded runner for PHP. They aim to run for any testing framework. Oh, OK, we're back. So scale the tests, and then scale Selenium. We're going to talk about the Selenium grid. And then scale the test environment. And I would say the hardest of these is a third one, your development environment. And the reason I mention this is scaling the test is pretty straightforward. There's things that'll just break it out into 30, 40 threads, and you can pound at it. There's third parties that'll help you run multiple Selenium on the grid, like Sauce Labs. I'm sure a lot of people are familiar with. But after that, you've got 60 tests running at the same time. But if you don't have a test environment that can handle 60 tests pounding at it, then it's all for nothing. That's your one bottleneck. And so I would say, if you're dealing with this, you can scale the test to a certain point. But at some point, you're going to need to scale your development environment. Taking your one environment, maybe putting on four machines and then doing a round robin. If you have the wherewithal and the resources to put in a load balancer in front, so it does it for you. But just something that either loads it out, or you can spend the money and buy like a 16 node cluster or something like that. I think it'd probably be easier to scale it out. So first, we'll talk about Paratest. And I chose the three-headed monster because anytime you have threads, it can be a little scary. Paratest is a multi-threaded runner for PHP unit. That's the only one I've found that actually works well. Because if you have a dependency, for example, and PHP unit has the ability to filter on one test. So if I do a command like PHP unit, and I want to filter, oh, sorry, thank you. Yeah. So if I do filter that, then it will only run that one test. The problem is, if that test has a dependency that's in another file, it'll fail. Because PHP unit isn't smart enough to go and load the other file as a dependency so that when the test runs, it runs a dependency and then runs that test. Paratest gets around that because it looks at all the tests and it kind of knows the dependency tree. So when it goes, and if you think about if you're going to do this yourself, the first thing you would do is find out how many tests you have. Put those into different threads. Make a thread stack and put them on each one. And that's when you run the dependency problem. Paratest basically does that. But then it's smart enough to load it the correct way to say, OK, this test has this dependency. This file needs to be loaded with it. So then that test can go out and be run. Anyone see my mouse pointer? There it is. So I do like Paratest. And I don't really have other suggestions for doing it. It's a successor to, Sauce Labs started it with another test runner. And then this is kind of the successor to it. And it's pretty good. Next is the grid. SlimGrid is a way to, for example, I'm running on my local machine. And the grid puts it out onto multiple machines. So I can then take six different computers and each one with different configurations and run them from one area. And when I run my tests, it puts them out on different machines. It's also smart enough to know that this machine only has Internet Explorer on it. This one has Firefox and Internet Explorer. So when you send it, it sends it to the right machine. And I can go over the grid, how many people have worked with the Selenium grid before? OK, so I'm going to do a quick example of both the grid and Paratest. So the first thing I'm going to do is I'm going to shut down my standalone server. Oh, you guys can't see that, can you? Let me turn on mirroring. OK, the first thing I'm going to do is, so I've shut down my standalone server. And what I'm going to do is I'm going to start a hub. And all this is, it's the same JAR file. And the nice part about this is everything, Selenium 1, Selenium 2, Selenium grid, it's all in the same file if you get the big one. So anyway, we run the JAR file and we just run it as a hub. So I'm going to start the hub. And then I'm going to start two nodes. And you'll see what I've done here. I just changed the role to node. I told it where the hub is. And I also told it that the maximum number of sessions is one, just because my laptop can't really handle a lot of things. So I'm going to start two nodes each with a maximum of one session. So you can see that the grid is going to put one test on one node and the other test on the other node. So I'm going to go ahead and start that up. I'm going to start a second node over here. I'm just giving you different port numbers. That's what that number afterwards is. So what the nodes do is they start up. And I told it where the hub is. They pull for the hub and they register themselves. And they also, all this information here, is its capabilities. So the node will tell the hub, these are my capabilities. Running out of time. OK. So you can see it here. And you can see I've got two remote proxies. And then I'm going to go ahead and run paratest. Just make sure I do the right one. I'm telling it just to use two processes, because I don't want to kill my computer. And then the dash f just means functional. By default, it does suites in each thread. And I just want it to do each single function in one thread. So we don't really care about the test running. It's going to run it and we're going to see it pop up. So there's one and there's the other. And then if I go to the grid, you'll see these have dimmed out, because now it's saying we've got one concurrent test. And then it's just going to keep going, because now it's splitting up between the two. And you see this is one node and this is the other node. And it's splitting up between the two as the test come in. So that's how the grid works. And you can scale this out to multiple machines, as you can guess. I could do a tunnel to my remote machine that's headless and put test over there. So then you can start to scale out your tests. So now we've got paratest running in parallel. We've got our grid set up. And obviously, the last thing is after you scale that up, you've got to scale your development environment, because if you've got 30 threads running at the same time, you've got to be able to handle that kind of load. And I think I might be out of time. I want to leave some Q&A. Does anyone have any questions or comments or find to? Yes, with contact information? About this one? Oh, yes, yes, yes. It can be used. Again, with the drawbacks of WebDriver, I also should have mentioned like anything, there are some bugs. For example, in Chrome, and I know the specifics, but one of the event fires isn't perfect. They're working on that. And you'd need a separate if downloaded driver, because like we said, each browser uses its native driver. So you have to download a driver for Chrome to for it to drive to be driven. But I would say it's ready to be used, definitely. They've already deprecated Selenium 1, so Selenium 2 is where they're going or WebDriver. But it's like, again, not a magic pill. There's like anything going to be some bugs here and there and some growing pains. But I'd say it's ready to be used. Yes? What's in your opinion? Well, again, so with the BAHAT talk, it's really, I think it should go to it. I think for me, I take BAHAT as a different animal, because this is a testing tool. And BAHAT is almost something like, especially if you have an organization, a lot of buy-in. Because you could, for example, you have a user story and you have someone who wants something done. Or even if you're a consultant, you give them the functional paragraph. Like, giving the user is logged in and it's 8 o'clock. Give them a lottery ticket. And then you put that functionality. You give it to your client. The client's like, yes, I sign off on that. And that contract is now kind of living breathing, because now you have tests that adhere to this exact contract. And so it's kind of a more, I take it as kind of an organizational thing. I mean, you could use it just as a peer QA tool. But I don't really see much benefit of that over all the other things that are out there for QA tools of just driving a browser based upon. Because you get the specs. Like, OK, the user needs to log in and look at this. And like, OK, well, you translate it. You put it there. But you're always doing a proxy in between you and the person who wants the feature. They're saying one thing. You're implementing the test. And you might have to tweak. I was like, oh, I meant this. But when you have this natural language that says, given the user is logged in and it's 6 o'clock, or you specify more like, oh, actually, I need the background, the lottery ticket to be gray. Given the user is logged in and it's after 8 o'clock, given the lottery ticket, that is gray. And now you have, OK, now we have a contract that says, this is what we want. So to me, they don't purely, totally equate on a technical level maybe, because you can write things under them. Like, Bahat can use web driver, waiter, anything underneath to drive the test. It has a pure HTTP tester that'll just do curl commands. So it doesn't even fire up a browser. So if you need pure speed and you don't need JavaScript, it'll just do that. But I think with all that architecture, I think there's something else there when you're talking about natural language and whatnot. So that's my personal opinion. I'm sorry, but we'll have to end here. We'll continue the testing thread in the subsequent sessions. And of course, Ernest is available to you for personal talks. So, but we'll have to end the official session here. Thank you, Ernest. Oh, thank you. All right, welcome. And you also have some simple tests. Yeah. Well, simple test covers the need for functional tests. But for integration tests, for example, AJAX forms and full stack integration things, you need something like WebDriver that actually launches a browser that does pop-up windows. Because simple tests has a very simple mechanism for it does basically curl commands when it does forward posting. So if you need to test that when I hover over this text box, this pops up, you can't do that simple test. And so you need something like a slender. But on the Selenium driver and PHP unit, you can't do anything. It doesn't cover all what you can do in a single test. Say it one more time. The PHP unit and Selenium can cover anything that you are doing with simple tests. Yes, it's far better. It's far better. The migration to PHP unit testing is because much better. So why didn't you go all the way and just eliminate it? Because if you're going to, for example, if I need to just test an algorithm, I need to make sure that this does 1 plus 1 is equal to 2. I can just test that function in simple tests very quickly. Where like you saw the subtle in this, it's got a fire up that. I've got to wait for the browser to pop up. So it's speed, and it's just kind of overhead. But the other thing is that I've got a talk that goes, I'm not doing it here. But basically, you have unit testing, which is things like functions you're talking about. And then you have the UI testing, which is a different level. A lot of what he was showing is that higher level testing, you still need to do all of the other tests. And you can do it in PHP unit. You can do it in simple tests. And the reason everything's migrating away from simple test to PHP unit testing is for standards. It's too complicated for you. I would definitely not recommend doing all your tests. And maybe that relates to the other question that I have. Is anything of that can also cover the need for testing theming level or UI? Yes, definitely. Because you showed only things like forms and functions. I wanted to keep the examples very simple. I didn't want people to get caught up. And I wanted to use the same example throughout. So everyone kind of could build off it. And say, I've seen that before, what's different? It was a good intro. It was really good. You covered basics, and you stepped through it. It was good. You got bit by Murphy's Law, like two or three times. And you also got to do the first ones. Thank you for taking the bullet for the rest of it. I'm glad it's over. Yeah, so I can bring that up. Oh, and that's what I was going to say. I went ahead and started trying to do the continuous integration piece. It became so long that I couldn't do it. I'm afraid why it doesn't feel bad on that. I think that's a huge piece. You needed a whole lap for that. Exactly. I'll talk to you later. OK. Yeah, so this. Afterwards, because I've heard that there is a kind of mistake. There is. But I don't know if it'll go into headless Selenium. So but yeah, it'll get you started. There's no display. So there could be a boxer, but it's running Firefox. It's doing AJAX requests and pop-ups. But we call it headless because there's no display actually. Mm-hmm. Yeah. Yeah, so yeah, the next VFB buffer, and then it just writes in the frames. So yeah, this is ours and all of our tests are. Yeah, now I don't want to steal your lecture, right? OK. OK. I'll be around all day. I have another talk tomorrow. So I have some prep for that. And then after that.