 All right, so I'm here to talk about experience that I had at a previous job where we grew from zero to 9,000 selenium tests So if you're looking at the conference guide, it says zero to 15,000 selenium tests. Sorry. I had an off by one error I'm a dot net guy the fact that I had any tests at all as a win So I'm a gym homes if you want to follow me on Twitter. I'm not the gym homes. I'm just one of them so the story is really about Going into an organization that didn't value automation. I had to figure out how to get some quick wins and Start to get us some good effective testing and then how to deal with a number of issues that happened along the way Went to this company. They had no QA department. They had no real testing. They didn't value automation I went in with my eyes open because I went to go work with a couple friends And I knew the ugly corners. So that wasn't the scary part there The sound of care. Do I need to move that down some? Good. Okay So when I first started things off, I had to make some quick traction To make the case for automation and show that there was some value around it So I used selenium IDE got a few test cases running there about a hundred wrap those into our build process and got them running three or four times a day So that we were starting to catch some regressions because there were plenty to be caught and People were starting to see the value of that my QA team grew. I got a couple developers We started writing some tests in selenium one because selenium IDE is totally inappropriate for any kind of larger Anything other than just sort of a proof of concept and we grew to about 1500 tests in selenium one Oh scattered over about a hundred hundred and fifty fixtures Selenium one we're having to deal with selenium RC. So it's a separate process. You have to stand up Selenium one didn't deal with page weights very well. So we moved over to selenium two using web driver And this was a big win for us because we got to get rid of the whole selenium RC Process so I didn't have to worry about it hanging. I didn't have to worry about it leaving orphan browsers around and At about that same time I Got another guy on our team and he helped us writing some backing API's He knew the platform that we were working on and he could help us out with things like very quickly creating setup data Creating prerequisites and conditions. So instead of having to use the browser to do all that which is just terrifically slow Now all of a sudden we've got a backing API So we're able to write tests much more effectively much more clearly and much faster So my two people that are doing the automation are starting to make some good traction on there I go away for a couple months Getting some focus on some other areas that needed my attention and I come back and they've got 9,000 tests Holy cow. So this is actually great because they're all high-value tests. But the problem is They're taking forever They're taking 16 hours to run. So instead of being able to get those things running Four times a day. We're having to wait until the weekend So instead of having any kind of a reasonable feedback loop all of a sudden it's seven days So they start losing credibility because nobody's paying attention to them because we're not getting any kind of quick feedback Functional tests are never going to be anywhere near as quick as your unit test or even integration tests So you just have to kind of buy off into that going forward But a one-week feedback cycle is just totally a mess so a Couple different ways that you can approach trying to solve that is Basically break things out into basic validation tests little small pieces that you can run on a much faster basis So we did that we looked through our eight or nine hundred fixtures and we said okay These are the really critical things that we need to run several times a day to make sure that we're not Completely screwing things up. We pull those out run them on a separate build cycle That takes about 30 or 45 minutes and we're getting some better feedback But we're still losing that comprehensive testing and the real answer to that is scale out Scaling out means taking a look at tooling that will allow you to run multiple tests in Parallel and then stitch everything back together And you should address this early on but here's the deal infrastructure is important and Infrastructure in many organizations as all is a battle I need more servers because I need to be able to run these tests faster rather than just once on the weekend It doesn't have to be going out and buying fancy servers. You can look to do this through virtualization You can spin up a number of virtual servers Look to your tool set to spin out those tests into those scales or into the multiple systems And then stitch everything back together a number of build server products So help you with that if you are running Selenium You can look to Selenium grid that'll put those out to those agents or those nodes and people will write this kind of Tooling themselves depending on their environment, but that's a really critical piece and frankly I screwed up as a leader of that group by waiting way too long to get that done I encourage you to look at the scaling early on as you're starting your effort Get that thing wrapped get it nailed down And then you don't have to worry about it later on when your tests are starting to take longer Add more nodes in add more agents in but approach that early early on So the next thing is And this was actually one area that I was pretty successful with my team Treat your test code like production code because it is dummy And that's targeted at me not y'all just to be clear So in our organization the people writing the Selenium test were not developers per se They were testers who had some coding skills And in those kinds of environments a lot of people a lot of time people don't understand things like Refactoring or keeping your tests dry but this is really critical stuff Constantly refactor it as aggressive as you refactor your production code The system you're delivering you need to do that same thing with your test code Get rid of tests that don't have value anymore keep things clean And dry is critical to find your locators things like ids how you're finding your elements that You have to interact with put them only in one place because if you don't all of those same Ugly things that happen in your production code will kill you in your tests as well The tests get very brittle and Selenium just by nature any UI test by nature is brittle And it's sucky enough already so don't screw yourself with overly wet test code So the next thing to focus on to help out is backing apis One common test case that we had to deal with was User b can reply to a forum post that user a had created So the test case for this required me first creating user a create user b Create a group to hold the forum create the forum Create the forum post and then I can finally get to the test which is go find that forum Post and reply to it and make sure that that works those five steps Trying to do all of that through the UI Add tons of time on your testing So push all of that off to factories baseline datasets or write yourself a backing api to get all of that done through your internal apis When you start doing this stuff very quickly you can Cut down your execution time and you can make your tests much more maintainable and get better tests going Focus on value we talk about lean and we talk about focusing Value on the systems we're building and that's a great thing because it keeps simplicity in place You have to do the same thing with your tests There haven't been many bullet points in the presentation. So I thought I'd just throw these up in case you wanted some Value is important and you need to focus on it in our particular application Crud tests were the really important thing for us. We had a complex security system We had a bunch of different roles And so crud was the very important thing that we needed to make sure everything worked on so fancy stuff like Validating layout maybe doing some intricate messaging tests It made no sense for us to spend the time writing those tests or running those tests So by staying focused on value that helps cut down that execution time So something else is look at how your tests are running because you need to keep them granular So in my example of replying to a forum post I get all my prerequisites in place and now I spin up my browser to actually do the tests that I need If I have to first drive that browser to log on as that user Then navigate to the forum post that I need to reply to In our particular environment that took 10 seconds So we had about 800 test fixtures or times that we needed to spin up the browser and do stuff 800 fixtures times 10 seconds is 8,000 seconds 8,000 seconds divided by 60 seconds per minute is about 133 minutes That's two hours out of your test run Doing stuff that you don't care about So figure out ways to creatively get around that kind of stuff in our case We were going to make use of cookies so that when you stood up a selenium browser You can pass the pre-generated cookie and the system could see that the user was already authenticated And i'm able to bypass that whole log on stuff that log on stuff needs to get tested But it gets tested elsewhere in very granular tests Skipping that lets me focus on can I go find that post? Can I click a reply? Can I put tests? Can I put some text in click submit and it shows up? That's my test Be very careful about what you're testing when you're doing functional testing any test, right? You don't want to pollute with things that aren't germane to that specific test So culture Matters no kidding there are entire conferences about culture. There are huge books about culture There are any number of people Including myself at conferences who hang out and drink large amounts of alcohol to try to deal with culture issues Yeah Been there feel your pain So if you don't have a culture that values testing and that values automation You're screwed How do you deal with it? I went into a company knowing that they didn't have any value about this But that I had a few allies and I had some people that were open to things So I started gaining more allies and I started gaining more Openness toward automation by showing small winnable things So I talked about using selenium IDE at the start of the project In order to what give me a couple quick wins I spent a couple weeks got some tests up got them running regularly We were catching regressions and my boss starts to see oh, this is worth the effort I can get you some more people to help out with this and we can see where that goes Doing that kind of thing starts to open people up to automation. The next step is Testability how many folks here have worked with? We'll say legacy uis that aren't necessarily the most test friendly Liars, okay. There's hands going up So in our particular case The platform that we were building Was at that point about six years old And so we're dealing with table driven layout nested table driven layout Tables within tables within tables in some cases like four levels deep gratuitous use of divs for things that we couldn't figure out And basically like one id on every page, which was at the top level of the form So when I have to try to go interact with a specific Field or something on the page more often than not The closest id value that I have is about six elements up So I am basically screwed and having to resort to x-path Does anybody here like x-path? I'm really sorry. There's therapy available for that It's not all about me So x-path is a large reason why at one point I was drinking about a bottle of scotch a week, but I had to get over that So if you're in that kind of situation look to solve some of that culture issue By getting closer to your ui developers in my case Every time that I'd go to a couple particular ui developers and I'd try to say look This is a really difficult ui to test Can you get me an id value that's on this particular element so that I can Have much faster much less brittle tests I'd always get a huge number of excuses why they couldn't do that I never got really good at winning that battle But what helped was getting a couple of those ui developers to spend some time each day with me writing tests So that they understood the x-path pain I was having to go through And it wasn't a hundred percent solution, but it was a step in that right direction I don't even pretend to have fixed it magically while I was still there Um unicorns were being killed on a regular basis, but you have to at least make that effort Because if you can't get that supportive culture in the supportive environment, you're never going to get over those hurdles so Kind of wrapping things up Long-running tests are tough and you have to look at how you can Get them running more effectively Look at running subsets those tests break them out into basic validation tests so you can get a much faster feedback cycle Look to do things like scaling things out get your infrastructure in place and get it nailed down early on Virtualization is a great thing. You don't need huge servers To act as agents or nodes for functional test stuff You just need to stand up some kind of a workstation A virtual machine without a lot of resources and be able to split your tests out So if those things run in parallel, let's solve that early in your cycle Treat your test code like your production code. It is production code Refactor mercilessly Look to do things of all of the solid principles as far as i'm concerned in functional tests The dry principle is your biggest ally Make sure that your test automation writers who may not be Regular developers understand the benefits of that Focus on tests that are extremely valuable Keep them granular and look to get rid of any actions in your browsers That don't specifically pertain to the thing you're trying to test And by all means and this is the toughest thing out of everything we keep hearing it again and again again It's not the technology. It's the people Look to try to address this Try to figure out how you can bridge some of those communication gaps when allies So i'm a gym homes on twitter. I love talking about this stuff. I'm very passionate about it If you don't hit me up here, you can reach me through my blog at frazzleddad.com. I have two small kids Which is why I have gray hair So that's it. I got a couple minutes left and i'd be happy to answer questions No questions for the dot net guy. Sorry All right. Thank you very much