 All right, cool. I guess it's about time to get started. Can everyone hear me? Okay. That's usually not a problem. Okay, cool Wow, this is actually pretty full house for 5 p.m. I'm I'm impressed so Good job everyone Hope you're enjoying Drupal con Austin This is pretty exciting for me because the last few years I've lived in Austin So this is a finally one on the home turf If you're here to talk about automated front-end testing you have found the correct room If not, I'm sure there's something else interesting For you to roam the halls and find but that's what we're going to talk about here I had a talk at Drupal con prog called front-end ops and it was just the process of automating front-end stuff in general and At the bottom I had called it or how to automate the process of breaking things So this is the how to automate the process of detecting when you break all the things That's what we're going to talk about here Cool. This is all working good. I am Chris Rupal. I'm a front-end developer at four kitchens Four kitchens is based in Austin, Texas here But I live in Germany right now So I just moved a couple months ago, but I work remotely for them and if you're looking for a cool company that You want to work for and you're not necessarily living in Austin we are hiring There's a link underneath this logo at the bottom The slides are online on the session description already so you can get them there. Oh There we go So some of the things I do in addition to to working and building things for clients I like contributing and presenting about open source in general. I'm not just Drupal I have this cool talk about CSS Transforms in there if you want to look at 3d in the browser I maintain the modernizer module for Drupal I Contributed heavily to the fast click Drupal module to like get rid of that tap delay on all your mobile sites go downloaded And we're working on a pre-rendering prefetch module for Drupal to To do some like speculative rendering in the browser, which is me I also posted a Jekyll schedule thing if you if you build Jekyll sites I know that's borderline heresy here, but I've already gotten enough Grief for it. So I don't mind talking about it Yeah, Dekyll's cool, too. Yeah, check it out Dekyll with a D Amatai from Gizra, I think the name is Built it anyway So the difference is from server side testing. We're gonna talk about testing the front end That front end might be Drupal. It might be something a Layer that sits in front of Drupal at four kitchens We do a lot of work with decoupled systems and we'll build like a node front end that serves the website But Drupal is the content management system behind it So all these things that happen on server side testing have to happen in a way on front end, too So if you're familiar with server side testing a lot of this won't be new to you The front end however is very different from the back end, you know You're talking about one server that serves up pages to many browsers now in this case We're talking about testing the many browsers that are connecting to one server So you've got many environments to test and There's a lot more fuzziness involved Often on the server you'll get a 500 you get a you know a Stack trace from Xh prof or something like that And then on the front end you don't exactly have that you have your javascript console But there are a lot of things that we we humans consider errors that a machine cannot consider an error There are minor CSS changes that throw the layout off Changes to javascript files that like during the build step it gets broken somewhere along there Just because you forgot a semicolon at the end of one file Or maybe your aggregates are changing when they shouldn't be because or have you ever looked at the Drupal aggregates? there's like that one little C tools aggregate it drives me nuts and You know it should just be like piled in with everything else And so you sometimes you want to monitor if those things are creeping back into your to to your site There's also like bigger performance regressions maybe Oh, I'll have an example later, but you know you might upload content that changes The website and and causes a regression. It's not just like a development mistake So the front end development Trade is becoming more matured, and we need better tools to help us check all these things because people rely on the front end To do a whole lot more than they used to it's not just you know center tags and marquee anymore So we need the same testing abilities that are have been promoted and Tested and and and tried on the back end for for many years We need to be able to test page load times. We need to test the render speed of the browser If you're developing a site that has a performance budget, you need to actually stick to that performance budget You might want to verify that visual changes have occurred or perhaps verify that visual changes have not occurred In in other places on the website sometimes you want accountability for code changes and So there's lots of things on the front end that we need to pay attention to and and you know you can't just React to it you want to be proactive and you want to work towards this goal with it in mind in the beginning, so I Want to talk about altering our workflow a little bit here and show you lots of tools that can be used by teams To help facilitate this change and to facilitate you know more reliable Development So there's a quote by Ilya Gregorik of Google. He's a web performance engineer and he's Amazing his he said performance is not a checklist. It's a continuous process. So You can't just think about it at the end a lot of my job has to do with front-end performance So there's going to be a bunch of those tools in the second half But and while there are things you can do to fix it after everything else has been done You know sometimes It can't it can't quite be fixed the same way as if you'd been thinking about it all along and had kind of baked in the Performance requirement to that that feature that took you know three or four sprints to build Another one is don't take measures without measuring them So you often see a lot of best practices thrown around on the web and I am not a believer in that term There are good practices, but they're The web moves too fast for best practices to stick around for very long so test test test and test again and get some data and measure it and then Do something about your measurements? So That's the if you took away nothing from this this whole talk. That's the most important thing So yes, these slides and the code are on github their open source as with all respectable code And you may take any of the examples from the slides and use them for any purpose commercial or not You are free to do that and I would be delighted if you did and if you find a problem with them Or have an improvement to make to them I would also be delighted if you contributed it back on github that makes everyone have you know a better set of tools so and like I said You can go to my Drupal con session page. There's a link to the slides already up there Feel free to to follow along on your computer or trial these code samples for yourself The Wi-Fi here is actually pretty smoking. So I'm Optimistic about all these being impressive up here. So functional testing functional testing is the act of making sure that a piece of Like a feature on your website actually functions There are various levels of this But most of the time I think we're all familiar with the concept of qa So you might have to do it yourself. You might have a dedicated individual on your team You might have an entire department within your organization that handles qa So the quality assurance of the work that you're building and this involves making a human sit down load the website follow a set of steps and Repeat this process over and over as a feature is built and you might add more steps to that but a human is still doing that Browser technology has come a long way and we now have Really awesome tools that can reliably do this for us. So you can script the process of functional testing In many many many cases It's not a silver bullet and you may still need a human to look at a lot of things but For many basic tests we have tools to do it So enter Casper JS Whoops, sorry about that Casper JS is a tool built on top of phantom JS. I try to put a warning in my slides I'm not going to cover every tool that goes down It'd be it I couldn't get to the bottom of this if I described every tool that we're building on top of here But phantom JS just to give you a one like an elevator pitch is a headless webkit And what it does is it does everything that webkit does when you load a web page like Chrome from a year ago or older or Safari That is the webkit rendering engine and you can using phantom JS. You can script Actions in web kit and you can do all sorts of things anything that a human can do and really more So you could run the same test You could run a script to check for the conditions on a web page at multiple screen sizes You can test multiple you can test complex features or navigation paths on your website you can test for the absence or existence of a particular element and You can count things you can do anything pretty much that you can do with JavaScript You can do with phantom because it's written in JavaScript All right, so again, you can even like log in to your Drupal site and automate complex actions. Oh, awesome So I'm actually going to do these on the console So I'm not even going to bother resizing this but we're going to run two Casper tests here And I'm going to show you how they work. So I will jump over to here and actually Show you these tests because one thing that when you look around on the web, you don't exactly see any Scripts that are fully commented. So I wanted to give Some material here that has everything explained within the file You don't need a blog post to go along with this thing. You can read. There's links to every single API call But what Casper does in this script, how many people are familiar with picture fill? I'm sorry So let's just get a show of hands. All right I'd actually like to see more because the picture element is going to be native in the next version of Chrome. So Beware So picture fill is a piece of JavaScript that simulates the picture element And what this means is that you can embed responsive images into your browser into the web page This will be native functionality in some browsers very soon But it in the case that it isn't this is what the polyfill is for But we need to make sure the picture fill Operates correctly when we build a website I have built responsive images many times on a site and we would write some code write some code and then I'm you know, maybe a few days later someone says hey The responsive images broke and you need to fix it and I'm like no, I don't think I don't think it broke I never do anything wrong And So then you got to have like a discussion about that, right? Well, there's no need for a discussion because we can test this all the time And so you can write a script that does this so what happens here is that I'm going to run five tests on the picture Fill that's what this five here stands for And you'll see that you can go through here and it opens a URL It checks for the existence of an element So it asserts that there is a picture element on the page. This string is a Like a jQuery selector, you know And then we can make sure that there are three source tags so that You know if you ran this test without checking to make sure that the things you needed were there When it's not such a good test and there might be something else wrong So this is again a selector that looks for three source tags inside of a picture And then we resize the viewport Then we'll go along and look for Inside picture we're going to look for an image tag that actually has a an attribute of medium.jpeg And so if we don't find that then the test the picture fill is not showing the correct image at this viewport We'll resize two times more and then the same thing will happen. So I'm going to jump over here and go to This example and I'm going to say Casper Casper So I'm going to run this and it's not going to take long because our Wi-Fi is awesome here great So that happened before I can even finish my sentence about the great Wi-Fi So you see that it says testing picture fill it says pass the picture element was found past There were three elements inside the picture called source and then it found medium It resized to 960 by 640 it found large and then it Resized to 1280 by 1024 and it found the extra large jpeg So that is That's basically all it's doing. This is a pretty simple test But it allows you to verify that this script is doing what it says it's doing on the page And even if you implemented responsive images at the beginning of your project and then four weeks later someone you know came and accidentally Included some code that broke your JavaScript aggregate and it didn't execute all the way Well instead of finding out after you've merged the code You'll find out immediately because you can have someone run these tests before they Before they end up merging the code into your code base, so it's pretty cool in my opinion And this is just front. So this is just this could be used on a static website This could be used on you know anything at all pick your CMS. So That's basically a sample script and that's testing the functionality of picture fill So this this test was successful. Oh, yeah, the next one is Drupal. So We're going to run a Drupal test now. So we're gonna run Casper JS test Drupal JS And this one actually I'll show you the code real fast So this one's a little bigger, but there's this website that I I just used It's a demo open source CMS comm slash Drupal you can go to it and like mess around and it will reset every 30 minutes it publishes the username and password so I took it and then I've got the sample node that I'm going to post to the Drupal site So the title is going to be hello world and the content is going to be this content was added by Casper JS Testing doesn't have to be creative. It just has to be functional, right? So it's going to test the Drupal site. There are eight tests in total that this one is going to run And you can fill in forms with Casper as well. That's what this particular Line of code does here and you can store the form username and password or passed into this form You can run comments just so when your functional test Executes a step that might take Longer than you're expecting or longer than instantaneous. You can issue a comment in the log that says hey This is what I'm doing. So if you sit there and it you know hangs for four seconds. You'll be like, oh, it's logging in cool And then we're we can do some crazy stuff, too. You can check the HTTP status And you can check this is checking 200 200 it is the status that occurred whenever you see a web page and nothing went wrong So 200 is the one that we always want to see you could also explicitly test for a 404 or a 403 Or a 301 if you want to make sure that your redirect module is working correctly You could even test for a 500 if you've got a page that deliberately breaks your site in some way And then we look for this logged in class that we're all so familiar with when we are working with Drupal sites We go ahead and click a link to see the content list the overlay will be enabled and so Then we are going to go and check and see that the title changed and also we are going to assert that the path changed In accordance with how overlay is supposed to work We will then again click another link and go to the node add page page and Then we will add a basic page We will go ahead and make sure that loaded correctly fill the form with the content that we looked at at the top of the file and Then we will say that we're saving a new node because once again this might take a moment And then we check and make sure that the title matches a regex of the title that we submitted And then also we would check and make sure that that node contents that I showed you at the top of the file Actually matches, so and then there's this little block at every bottom of every Casper test that just makes it actually run So I'm going to go back here. I'm going to hit enter the Strupal isn't in front of varnish, so oh Awesome fail Well, there you go. Sometimes your tests fail. That's not a big deal Hmm. I don't know but when your tests fail it tells you how to do that I'm not stressed out enough over this test to like actually make it work in front of you guys That's not the point of the exercise, but sometimes when you actually get a failure You know you're gonna see hey something's wrong Sometimes it'll be your test and that might be what's happening here. I ran this five minutes before the presentation and it worked Who knows? Yeah, it could be a network issue, but I Don't I don't think it is because the other one was using the network as well, but still You can go ahead and look at this and probably run it on your local and maybe someone will have more success than me But you have an incredible set of tools at your disposal with Casper to do all sorts of things Anything that you would ask someone to go and click around on the website and verify you can probably verify with Casper So we'll give it one more try and see if I see if it works this time See if anybody have any questions in general about about Casper Cool if you have a question actually could I ask you to come up and talk into this microphone real fast? Look at that. It worked this time Yeah, this does Casper allow you to record tests at all or do you actually just have to write out the script at this time I'm pretty sure you have to Write your tests yourself You might be referring to like Selenium has a an option where you can use Firefox And and it will make a macro out of something that you do on the website and it will record the clicks that a human makes That is an option and there are other tools to do that. I don't believe that Casper does that Thank you. Yes. How time-consuming do you find it to be to write all these tests? Not very time-consuming at all if you are an Intermediate jQuery person who actually writes jQuery code you can do this If you just like install a jQuery plugin and you like run the init script That's not what I'm talking about you have to write code But if you're familiar with JavaScript enough to write your own code Then you can probably write JavaScript to test your code, which is awesome And Casper can do also it can do unit tests So you can just mock an object from the page and and take it and like make sure that your library is functioning Correctly if you wrote one two, so If you got a question, I would ask you to step back there. Thank you. Yeah, no, no problem I'm gonna look at the Yep All right, so I'm gonna go to node one. I assume that I'm the only one messing with this thing. Yep, so here you go This says hello world and it says this content was added by Casper JS So this was the node that I published in that successful run So that's kind of how it works. It said it was logging in and adding the node and all that so you got proof That it actually did what it did Cool Next question similar to the first one. Is there actually a way to capture what happened? Visually during during the test. Yes, there is phantom can do that and there are As I mentioned before I can't go into the extreme details of of how phantom works But I do have a slide deck on this Drupal con session For a past session that details how to do that In quite in great detail, that's what detailing is So is this Doing the same thing selenium would be doing would you be using Casper instead of selenium? That's correct You would use Casper instead of selenium or possibly in addition to I'm not really sure if you have one in your infrastructure It's probably easier just to have one, but And I know that selenium's made great leaps and bounds actually the last time I used it was Quite a while ago, and I was not it wasn't in JavaScript. So I Hear that that's how it works now, too. So this is just another option That is similar to selenium cool. I have a second part, too I noticed get element by ID, but you just mentioned that it can use jQuery. So Well, there's a catch also When I likened it to jQuery, I said it's about as hard as jQuery and You can Use jQuery in your scripts. So you can see here that this says return jQuery node page content P I can do this and I made a note about it in the comment here That since you're testing a Drupal site, and I know that Drupal 7 comes with jQuery 1 4 4 at a minimum I can reliably write my test to and just use jQuery without checking to see if it's there And that's just how Drupal works and so we can make that assumption If the page does not include jQuery, you can't use jQuery. It will say that that variable is undefined So you can read more about the assert eval equals This function at the at the API doc at the docs here but this line of code is being run in the The environment that Casper created so this line of code 169 in this file is actually running on the Drupal site It's not running in the Casper instance So so you don't provide jQuery as part of your test it has to be in one of your testing you could You could inject it into the page You could do that. I wouldn't because it's not there and it might you know sure you're gonna cause some uncertainty there but if you have jQuery on your page already feel free to use it if you were testing a Ember app or something like that you can make ember calls and Go ahead and use them as if you were typing things into the JavaScript console of the web browser of the fully loaded app Cool. Thank you Hey, so we have a need to test chat a lot of times is it possible to fire up two different browsers and have them interact to test the You know JavaScript interaction of a chat program. You really need like chrome and Firefox. Oh, that's that's interesting Well, I gotta be honest. I don't have any experience running multiple instances inside one Casper test I'd have to look it up Can you can you choose which browser it uses or Casper? I Don't remember off the top of my head It's not slimer. No, okay Okay. Yeah, actually it does say that it supports slimer right here. So You can run slimer is a not quite headless It's basically the Firefox version of phantom JS They're not quite headless. Yes, but yet but they are aiming for feature parity. So Maybe it would be possible that sounds like a gnarly test. It sounds kind of awesome. So Yeah That's something to shoot for Okay, cool. No more questions Awesome, so that was functional testing where am I doing great So for performance testing like I said, this is a huge chunk of my job at four kitchens and I really enjoyed doing it Nothing makes me happier than you know getting to look at something that someone else built and telling them how they could do it better, right? So When you talk about page performance, there are many many contexts of this But the one that is pretty most important is page load time testing So how fast did the page load how soon did the user start seeing the web page because if it Takes too long then the user is gonna get bored. They might hit back. They'll go to another link We all know that story so There are a bunch of tools and I'm gonna go over a couple here Some of these you would use either or you might even use a couple of them on top of each other and I'll try to make note of that But the first one here is a Way to run automated page speed tests So Google page speed is a tool that analyzes your website and it checks all these different variables and Gives you basically it boils it down into a single number which you can aim for and so you might say as part of your Development goals you'd say we want to launch this with you know an 80 of a page speed score of 80 that's Just you know a way to make sure that overall the page is going to load fairly fast And so they call that a speed index So before you can automate page speed you'll have to log into code google.com and get an API key But they're free and they don't have any like you're not gonna run into the limit They do have a way to pay if you want to set this up for like thousands of instances or something like that But basically you can get this for free and get running pretty quick So grunt page speed and again I'm not gonna explain exactly what grunt is but basically it'll run tasks for you many tasks all in a row And instead of doing it on a web page like Casper did it grunt does that as if it was your command line user So instead of you saying, you know, I want to minify my JavaScript and move it into the you know Aggregates folder it can do all that every time you save your file and so grunt is something that you could run and like You wouldn't want to do this on file save But perhaps before you commit your work you could have it run a little grunt page speed test and see how your remote instance is Doing so I'm going to jump back over to the console here and run this and give you a demo So I'm going to run grunt Because I set the default task up just to do this for us. I mean this is going to test grunt js.com in the desktop Context The screen's just a little too small to make this pretty isn't it? So, oh, you know what I can do. I think I can just yep. I think this will work better. Oh, nope, whatever So that one happened really quick because Google caches these responses for about 30 seconds But basically you can see here that when I ran the page speed desktop Task, which is just the default one that I set up in the grunt file And so it goes to grunt js.com The strategy here is something that Google defined. I did not define this so they have desktop and mobile and that's it And so it gave us a score of 91. So that's pretty hot The number of resources is 30. It shows you it says that there's a six hosts that served all the files That's probably due to a CDN It tells you how many bytes were downloaded and how many resources were static Versus like, you know, a PHP script that returns dynamic content How many bytes were in the responses and all this so it gives you a lot of good data Just kind of this is bird's-eye view data And another cool section that page 3 returns here that I can't quite figure out how to surface in the actual web interface There is a web interface for this These set this set of numbers is a set of numbers that are just relative to each other and What it means is that you're it's the bang for your buck Value so in here, we've only got two numbers that are non-zero There is a six that tells you to minimize something and Then there's an optimized image, which just means compress the images that are on the page The minimize render blocking resources issue is a six and the other one is a one point five so It is approximately using simple math here 1.5 goes into six about four times so this number goes in about four times So it's going to be about four times as effective to fix that other problem Then it will be to fix the the images problem and I've had runs of this thing where I Saw like 30 on the thing so you get a 30 x Gain performance gain from fixing that problem versus fixing another problem So it can help you prioritize issues just at a at a bird's-eye thing and and this is built by Google like I said they have a they have a a A an Apache Thing I'm front-end. I don't know it's called mod page speed and wherever you put mod rewrite You put mod page speed, okay? I Need I need my I need my it up here Matt's laughing at me So basically they will they have a little Apache extension that will Essentially do a lot of this stuff for you and it'll even if you have a static website that doesn't have combined and minified assets It'll it'll do that on the back end and just kind of like spit it out for you, which is pretty cool There are a couple issues with Drupal sites I know theodore via Dala. He is well never mind. He's formerly of Acquia He had tried this out and wanted to evaluate it for You know their use and he found a couple issues with it, but still it's a great tool and Solves a lot of these problems like out of the box sometimes So I'm gonna do one more here With this grunt page speed Mobile so the colon just means it's like a sub-task in grunt. So the first one's desktop. This one's mobile You'll notice something different about this one It looks meaner so Just in case people cannot see the difference between green and red The first one was green and it signified that the score was higher than the threshold that I set And this one is lower. So we had set a threshold of 85 The first score was 91 and this score is 79 So it fell below our threshold and this is a fail, right? And and all you're doing is you're you're meeting your own goal in this case But still the same set of data is there, but you'll notice that in this one There are a lot more numbers that can be fixed. So Relative to whatever the one is going to be here the configuring the viewport needs to be done Maybe this isn't a responsive site. I don't know there are 24 ass or no, they're not 24 asses Excuse me There are assets that are blocking the render pipeline in the browser and fixing those problems would be 24 times as effective or maybe 2.4 times as effective as that 10 above It's going to be well now. This is crazy math, but like it's gonna be 16 times as effective as this optimized image thing so All these have this this index here and you can look at these problems and you was you basically want to tackle them Tackle the one with the biggest number first go to the next biggest number and so forth And so then it gives you this this error at the bottom so that's front page speed and and Like I said, it's built on page speed and you have to kind of figure out the ins and outs of grunt But I do have a slide deck up there to help you get through all those things if you're not familiar with them The next tool is probably my favorite it's Funtomas So this is phantom.js based and it's a web performance metrics tool That's a mouthful, but basically what it does is it provides a different facet of Metrics here, so it gives you a lot more data. It exposes a lot more data than then page speed does so page speed is kind of for They they have links that go to all these help pages on their website, but this one gives you a lot more raw data So there's a huge use usage guide and you should check it out So I'm just gonna couple run a couple basic reports here and show you what it looks like So So I'm gonna run this and it's gonna take a second because it has to fetch the web page. Wow. All right it has to fetch the web page and Go and run all these analytics on it, but it does have to wait and request all the resources. Oh I forgot to mention one of the coolest things about mod page speed is that it's actually run remotely off of their servers So you do have to have an internet connection to use it But it's not based the the score is not based on your Connection that you currently have as are some of these tools So if you have bad Wi-Fi, it'll affect the performance of this tool, but it won't perfect affect the performance of page speed It won't change the data results So here we got a lot back I'm not gonna go through all this. I would we'd be here till you know 8 30, but It'll tell you the number of requests and it'll tell you like all this data about it You can see time to first and last bite the amount of HTML that came down Whether or not there were Ajax requests and how many HTML files were loaded if you have ads this number is going to be astronomical The amount of the number and the size of all your aggregates and all sorts of cool stuff like that So it's it tells you how many jQuery sizzle calls were in there and it has like all this special integration It's very very cool And so then it tells you of these things that were there. It tells you like the worst ones, right? So actually you can run this and and I was practicing for my slides and I found Kind of a minor issue, but I found an issue with our website and so I was like, oh cool you know we're serving from like a dev domain on accident and You know I filed an issue and found it but you know a manual inspection would have taken a really long time to find That kind of thing because it was actually behind a redirect so But bond must just tells it all to you. So we're gonna run that again Let's see if I can do this without looking. No, I can't view for Okay viewport equals 320 by 480 and Film strip which someone was asking about Can we take screenshots earlier? Wow? Did that? We'll just open one. No, that one's gonna be boring 16 So actually here I get So it ran the same data here and the test was basically the same as before But I actually specified a viewport this time So and then we can look and I'm just using the OS 10 preview here But it took screenshots at each successive see that there's no images in this one now There are some images finally that little add thing in the left sidebar loaded So it takes a little film strip and takes screenshots as the web page is loading. So This is kind of unscientific, but also it gives you an idea of how long your Page is taking to render. So it's pretty cool and you can just attach that flag to any of your font-a-moss tests And you get it for free So the last one I will show you is I don't remember the number so I got a check again 20 okay, yeah, of course Assertion so you can assert different requests and this just follows any value that you've found up here Up here this initial list So I could change this to this string to test. So you just say assert dash that string right there And so I'm gonna say 20 because I know that's lower than the number that it's outputting and so now when we run this So new test I cleared my screen you can see here at the top that it failed my Assertion that there needs to be 20 or fewer tests on this web page So this is pretty cool, too And you can have tests built in you can have a little script that just runs this command in various formats for several You know iterations right but you get this data up front Which is great And So I'd rather know about this you know like when I'm committing the code Maybe you added a module and it adds like four CSS or like four JavaScript files or something you know what the what? You know I'd rather know about that sooner than later when you're like trying to fix it You're like oh, how can we get rid of this module now? That's not gonna happen because it's providing you know that pole on the sidebar You got to you got to You got to take those quizzes so That's basically fond of us. This is a Foundational tool that actually is going to power the next two that I show so does anybody have any questions about this one real fast Awesome either. I'm boring you or I did a fantastic job So we'll jump back to the slides here Grunt Fontamas guess what it does it does what I just showed you but it's automates it That wasn't a very good reveal. So Grunt Fontamas doesn't actually just automate it and I lied So what it does is it basically instead of just running it for you like page speed does it doesn't really do anything besides provide the data to you Grunt Fontamas provides details Reports and it lets you track these the metrics over time and it Generates its own front-end to do this It's super cool And the guy that made this is a wizard and it actually even will take multiple runs So by default it it will load the web page five times and do these metrics five times And then take an average of those so you get you know your min mean median max And and so you can see these and then additionally that's just one data point on the graph You can see it as it happens over time So let me tell you a little story. I can't tell you who this is about but I Had a client and we were having some issues with performance and it was basically my job to tell I was like hey guys This was this was a thing that I was focusing on it was my role in the project and So I was always trying to make sure that everything we were shipping was not increasing the the size of the the website and a lot of times they weren't There was just like difficult conversations that The accountability wasn't there for anyone. Let's just put it that way and so I said to myself okay, I'm gonna start graphing this every day and You can see that You know we're going along here for a month or two this this actually shows a span of about two months and One day I came into work and ran this command and save the data just like I always did Except for obviously those big gaps in there. I don't know what happened. I Came in and and maybe I drank my coffee and then did this and so I saw this enormous spike And I was like both horrified and also Delighted because I was like aha this finally paid off So I was like danger everyone we've we've actually accidentally committed eight megabytes of inline data into our CSS file I don't know how that happened, but we can probably go back to the commit logs and figure it out Someone had used the inline function of compass You know where you can like inline a font or inline an image and it base 64 encodes it and spits it out into the final CSS well, someone did that with like eight megs of images and So when I ran this test in the morning and loaded the graphs and reviewed them I was like whoa, okay Let's go get this fixed and I took a screenshot of it and send it out to people and said this is the deal Let's look through the you know the last 24 hours of commits and figure out Let's look through the last 24 hours of commits and see you know when this happened And as you can see we removed most of them right then and by the next morning We were almost back down to exactly where we were over the weekend Basically by the time I ran this the next Monday. It was all good So if you didn't have this and you didn't monitor this at least somewhat regularly You might just be like man our site became so slow, and I can't remember when it was I think it was sprint seven, but like I just didn't have time to focus on this and you know Look at the web page and figure out what happened, but having a tool like this Is really helpful. I've I've even talked to our team about Running this automatically on our own website every day and having like a public display public, you know the wall of shame as as we Do this, but I'm joking about the wall of shame, but also it sends an important message I think that you know if if our organization is focusing on the performance of our website and we're willing to share the data with you Glorious client We will do the same for you, and we will make sure that your website is performing the same and doing this great job And and and being the best it can be so that's that's why I like this tool so much It's a it's a really good one and we're also working on there I've seen some cool articles go around there was one from amazing labs that talked about the elk stack So it's elastic search log stash and cabana And so for instance grunt font moss just produces json. There's nothing stopping you from putting That json into this bigger visualization tool and running it alongside all your other logs. So Lots of potential here. In fact, I wish I could show you something that I finished that involved that but I So this is just the syntax for font moss you can go ahead and run it But actually, I think I've shown you the graph and I've shown you the impact that it has That that well here we can look at a we can look at an example Okay, so I'm just gonna run it once here gonna do its thing. All right, there we go So it went and downloaded it actually worked all five times. That's awesome So it ran the font moss configuration that I selected five times and then it wrote to this json file And then because as I said it generates reports over time It checked all the other previous json files to make sure that they're not gonna break D3 And then it just saves it out to a static site. So I'm gonna open reports index here and Maybe that happened over here. Yep. Okay, cool so these these graphs look pretty flat, but this is the front end that that Grunt font must generate. So this is again through gruntjs.com And let me scroll down to one. That's a little more interesting looking. There you go So whenever I did this a few days back when I was demoing to someone else It looks like they had some really gnarly redirect time. Maybe I was on bad Wi-Fi. Who knows? and You can see here that whatever they had on the front page is actually smaller now because the dot from May 29th is pretty consistent with the dot from June 4th And you can see here that you know all these graphs are here. Actually, this is vector. It's D3 So I can like zoom in and out This is pretty cool stuff. So once you get a lot more data You're gonna see the data points and then it lists all the raw there for you You can also suppress Any of these categories if you don't want to see them all You don't have to You can just see like three of them just three of them were important to you go for it make a smaller report But that's basically the idea behind grunt font must How how many people are familiar with performance budgets? Cool couple hands. All right. The idea is pretty simple Just like your monthly expense budget. You'd say like, okay, I've got this much money for groceries I've got this much money for you know my conference Drinking or whatever. That's that's terrible. That's terrible. I'm sorry. I shouldn't have said that But we need to you know keep track of how fat a website gets right and You can only do that if you monitor it over time like I said in the beginning This isn't just something you tack on at the end These are critical decisions that are going to influence how you check that code in the very first time and there's no going back sometimes Depending on your stakeholders So performance budget is a way for them to agree up front and we can all agree as a team and the Product owner and the stakeholders can all say okay. We said we wanted a fast site That was our second priority and somehow it got moved down to number seven by the time the site launched Performance budgets are a way of making sure that doesn't happen. You said you wanted a fast site. You're getting a fast site Grunt Fontamas is working on performance budget features that will record and visualize the fails So right now it kind of just errors out But if you set assertions in your configuration The way that I envision it would be like that big spike that I showed earlier would be a red dot You know and like a lot, you know highlight that log entry in the raw data And so you could see and maybe even if you've got the assertion value Show the line on the graph and say our CSS can never go above this size, you know, it can never go above you know 40 kilobytes or whatever you want to pick And so then you can see the lead-up to that to that goal being broken as well or not goal, but limit being broken So performance budgets, I think are going to be a bigger thing in the next couple years and the main thing that is Stopping them from being wildly popular right now is lack of tools You need testing tools and you more importantly need visualization tools so you can easily show someone. Hey, it's broken Grunt dev perf is another one. This was I Was so excited when I saw this one, but it was only like six days ago. I was like man Okay, let me make room in the slides for here we go. All right, but I got it under control So you get to see it too We're gonna go to that folder in the examples So you'll notice that I'm just typing the word grunt over and over again That means that it's super easy for all of your team to get this set up and running right so they can all do this You don't have to type in arcane command in there You just type grunt all the configuration is hidden away from people for the most part in a JSON file for grunt If you want to use another task manager of your choice, you're totally free to and I don't even need to name them because you probably know it If you want to know it This one does kind of the same thing you can see that the Fontimus execution started and then the JSON file was there And there's a couple more data points than than the previous one And then it says hey everything worked and I'm gonna go to the reports now and look at this index file And horsey no smiley. Yes. Okay, so this looks like a console output, but it's actually a web page And so this one actually lets you set a line and this one lets you see your goal a little more clearly Like I said, these are all different front ends for the same data on the back because they're both running off of Fontimus So it's just another way to look at it and this one actually issues a warning So in this case it's saying hey, there's 16 images without dimensions if you were looking earlier You could have seen that in the raw output of Fontimus, but You know you could easily overlook that if you were just scrolling through this big wall of data So this will issue little warnings. So I gotta say that grunt, you know, it got a 91 on the page speed Which probably not gonna get the first time you're on page speed You're gonna see something that is a little lower and kind of like breaks your heart or something, but This is this is just basically another Way to visualize this data and and this is showing that gruntjs.com is a very well-built site It's just static HTML first of all, but it's got a couple features that are JavaScript But these tools are all agreeing that grunt.js is built very well That's why I stuck with the same domain over and over again So you can see that they are offering some concrete evidence of the performance of the website So that yeah, this I can close this all right cool So that's grunt dev per those two were both built on Fontimus and there that's the image without a couple data points This last one is built by the guy Tim Cadillac who? He came up with the idea of performance budgets If you have smashing book number four it is a it is an entire chapter and smashing book number four I haven't read all of that book, but what I have read was pretty awesome. So If you want to pick it up Do so actually and I live in the city where smashing magazine came from so if you want to visit me come to Smashing comp in a Friberg so I should fly blog Sorry, I Got it. My girlfriend's gonna watch this later. So I have to say correctly So Grunt perf budget relies on web page test or which is another tool Completely separate. It does not use phantom. In fact, it runs on all of the browsers and it's a very It's got a lot of history and it is a very detailed tool That can use IE 9 chrome it is geo distributed you can Throttle the network you can do all sorts of stuff with it And so if you're familiar with web page test this uses the API and it basically sets again It sets some assertions runs the data looks at the data tells you what exceeded the numbers that you were caring about so I'm gonna go ahead and show you a quick demo of this So it's running the default and this is a little different because like I said, it's hitting the web page test API And this one takes a second because it's got to go do its thing and then it's got a report back when it's completely finished So there is a gap in there, but luckily I was first in line and the test is running And then this is gonna come back with some metrics just some really small ones because I just picked to to make it a demo So you can see again that I picked 1500 which is actually pretty lax the numbers that I've been seeing going around our 800 800 to 1100 is when it starts to become unacceptable And again grunt got a 693. So that's pretty good And then I intentionally picked 32 so that it's 31 requests that I knew about Came back and passed so you could change these limits for yourself and there are a lot of Different metrics that you can look for I'm just gonna pop over here and show you that it does generate a web page test Page it shows you the waterfall that they captured and the film strip and all of the content breakdown and all that and Magento, I don't know when I went there, but apparently oh, and there's some sponsors, too So Yeah, so that's web page test or I got one more cool thing to show you and I am running short on time So I'm gonna zip through it But I'd love to talk about performance budgets with anyone afterwards or if you want The party at the bangers with the pantheon and New Relic and friends so we co-sponsor that party That's where I'll be tonight The last thing CSS regression testing. Thank you for hanging in there until six. This is worth it. I promise CSS regressions who's ever heard of that no one nothing's ever gone wrong with CSS So having no scope at all It's like way easier to screw up than than anything You know you put a rule that you thought was only affecting one part of the page and you end up You know shifting everything that left all over the place But this is easier to prevent than you think The BBC made this tool it is called Wraith What Wraith does is it uses either phantom or slimer again those two Scriptable versions of the web browsers to take screenshots of two environments producing a visual diff of the screenshots So think of your your dev environment versus the trunk environment, right? And you're on a branch and you're doing your stuff and your website is there and then the trunk is there Untouched before you've merged And so what you can do is you can run this tool It'll take screenshots of both of them at multiple viewport sizes and it'll show you the diff So their example shows you how everything got nudged a little to the side and that dark blue This might actually not be visible on the projector. Yeah. Well, the dark blue is there But basically it's a diff of of the thing that changed and so I actually ran this ahead of time because sometimes it's a little finicky but Wraith is pretty cool. So I'm going to show you Final demo time's up, right? Sorry everyone Okay, so I'm gonna first show you these two tabs Can you see the logo moving a little? So if you didn't check that this this one is for kitchens comms is live right now. This one is just a local instance of the website that I have And so then when I actually go and look at the oh sweet, it's not big enough There we go. So when I go and look at 50% sure When I look at these the first column is the The production the second column is the dev right and you might not be able to see the difference but then when you go in here and And begin one of these images you can see that my logo is wow actually you can't see much except for that diff But it's kind of grayed out this projector has a like a really it's like blowing out the image But the rest of your website is is visible there And so you can see in the context of which part of the website was this diff occurring But I showed you the two logos right so you can see that This is the diff that is displaying and so then this diff is showed on the screen and you can run this thing before you merge your branches and one Although you might be like well, yeah, of course I did some work and I changed how the website looked right, but you can make sure that nothing else changed a good example is that we built a website and we like forgot about IE 8 for like, you know seven weeks, whatever and So then We you know we like we were about to launch it and they were like oh we checked this on IE 8 and it looks terrible And I was like okay So I went and I did a bunch of IE 8 fixes with sass and compass and stuff But what I did was because I was only making IE 8 fixes I ran Wraith on my two branches to ensure that in WebKit at least I was producing a zero byte diff So I was able to confirm that nothing changed while I did my other work Which is the true value of it, right? You might want to make sure that nothing changed while you refactor some JavaScript code and like change the way that you're Instead of doing JavaScript animation you're using CSS transitions or something like that so this is a great way to You know both verify changes and verify the absence of changes, which I think is pretty useful So I know there's probably gonna be a couple questions and I can take them But I think I got to get through here and oh, yeah, you can put all this in the Jenkins and put it into Jenkins And also well kidding aside, we we have a blog post that Is still alive and kicking from 2011 and it goes through every step required to Get Jenkins to push to your server when you merge on GitHub and the way it does this is get hooks so There's a couple ways like I showed you things where you wouldn't want the code to ever make it out of your environment Right, so you can use a pre-commit hook to make the test run before you can commit the code you can use a pre-push hook to run tests before you can open your PR and you could use another set of tools just to like a Pre-merge or post-merge to like go ahead and like run tests on your trunk environment afterwards So there's lots of opportunities to hook into your version control system and make this happen automatically There's a bunch of links here if you want to check it out My favorite one is a grunt for people who think things like grunt are weird and hard by Chris Coyer so Yeah, check it out and these slides are pretty live, so I update them as I see new things I don't like them being out of date, so if you see an error or have an idea file an issue Thanks for coming. Please Do rate this session it helps me make better sessions and it helps you guys see better content when you come to Drupal con So, thank you. I got a short link here. It's a j.mp slash a us dash testing That sounds so simple when I say it out loud But you can go to my session page and rate it there I really appreciate you guys coming and I suppose we can take some questions or you just come up here and we can all talk If you guys want to go Before before everybody runs off I've got something really interesting to that you should know about It's a tool that we're working on that's combining testing with documentation And that's turning selenium tests into walkthrough tutorials and you you can record them without using selenium IDE You can just record them in the browser just start a session and click through it if you're interested in doing that I've got bet it's an open-source tool But we also run a SAS version of it if you want to bet I invite just come to me and give me a name cards And I'll I'll make sure you get a beta invite for that and there's a session about that tomorrow Above session at one o'clock. So it's testing and documentation in one The answer is yes Check it out Yeah, wow red