 So Are you actually familiar with how much time do people spend on a page before they leave if they're not interested Your guess Four seconds up to four hits four seconds. Okay, so I've wasted ten seconds of your time If you don't like this like a front-end you're free to walk away to balance. So if not stick around, okay, so anyway I'm actually a psychologist by the way Dealing with websites and the internet for the last almost 10 years So my role today will be to actually brainwash you into properly using Split testing tools and and how to how to do the split testing on the on the WordPress Okay, anyway What is split testing so How many of you actually in the in the room have used any kind of split testing tools are dealing any anyhow with CRO stuff doing split testing or whatever Okay, about 20 30 percent something like that, right? Okay So anyway for the rest of the room because I saw that there's a lot of developers around I'll be pretty pretty Let's say marketing oriented. I won't be so much technical as the previous as the previous presentations Okay, so anyway my role today is to actually Try to to get you closer to split testing or a B testing or whatever And to show you how it's better than to actually completely redesign your website every XYZ years anyway as as it mentions traffic's traffic is Randomly split between the variations So you have one variation which is a control which is current current one Then you have variation a bc or whichever and you randomly split traffic between the variations. That's the classic Classic example of split testing and that's pretty much the wider final definition. So Even if it's called a B testing you don't have to have only two versions you have you can have as many And there's additional notes You don't have to use actually the split testing you could be using something called multi arm bandit That means that this the traffic is not split Equally all the time it is actually split equally for some specific amount of time amount of days And then the multi arm bandit Algorithm besides which version is more likely to win and in that case it forces It forces more traffic to that variation. That's really good for fast split tests, but you'll see After after a couple of slides how much time do you need to actually spend on split testing? And the test itself Now what is multivariate testing? Well pretty much the same thing With the only difference that you instead of oh, by the way, the presentation is Will is and will be on on on slides here. So you don't need to take photos, right or whatever Anyway, multivariate testing is pretty much the same thing as the split testing as the only difference is that you have a couple of elements on the page and you don't change the whole page or The the most of the most of it you decide to change only one element. So title image one color here on there and The point is that actually you make a couple of those changes a couple of those blocks of changes So you have three variations of try title three variations of one image three variations of one button And you combine them all so you get a one b1 a to b1 Etc. So you get a combinations, right? And all of them are shown to visitors randomly at the same time The difference between the split testing classic split testing and the multivariate testing is that for multivariate testing You need a tons of traffic because it would take ages to to actually test it this way and get statistically relevant results So Why do that? Because you don't have a hypo highest paid person in the room, right which decides and says, okay We will do that. We'll change the design like this. Why? Well, because I said so or different variation we have designer which has tons of experience and He knows what's the best for the website, right? No so You split test and you you just put his variation my variation your variation, whichever you put it on test and say, okay this one one But it's not as simple Test what people really do actually you have probably heard About the usability testing and about actually putting people next to computer and saying them, okay Just do something, right? Probably many of you which have built the website anytime in their lives have put their grandma or sister or whoever Who didn't see the website and said okay play with it and then Saw what the problem is so that's the usability testing Let's say one-on-one So instead of them saying oh, I would click here or I would do that you actually split test that and You don't have Guesses on what they would be doing you actually get measurement on what they would be doing Then you can get insights From those plate tests into some some kind of marketing efforts so you split as the title your split as the image and you put it on in practice later on in the adwords or wherever, right and Actually conversion rate optimization can be done without split testing, but it's not valid because It's mostly like guessing which then again is not as the first first point, right data bit beats opinions So valid conversion rate optimization needs to have Testing so optimize or die and thanks for the title to Craig Sullivan. Anyway We have two options one is to actually be like Classic websites and to say okay. We have a stepwise redesign, which means every xyz years We have a complete redesign of the homepage of any product pages or whatever and With that we can say okay Our design changed to better or to worse But you'll find it out only when you press the button, right? So you can be extra cautious and say, okay I'll reserve like 10% of the traffic to the new design and I'll see how it performs before I switch the plug Pull the plug out for the for the X design But you might be losing a lot of money so instead of actually instead of Missing the gains between launches you would be doing continuous redesign Now you might say okay, but it costs more. I need to constantly have a designer on it I need to Develop changes. I need to do that this or that yes It does require more resources But if you have a mindset of continuously improving your website and continuously testing actually It's better in the longer run you just need to establish a proper process and The return investment just won't be a problem in that case I cannot tell you how to do that. You need to figure out on your own But I get I can just say that websites which are continuously testing and improving Are performing better? Example Amazon. Hello, so they're constantly testing something. Did you know by the way that Google? The color the blue in in one of the oaths, right Google They did some ridiculous split test like Changed the variation of the blue color in that I mean, but they're crazy guys because they can they can afford The billions of people every day That could be testing a millions of variations of blue So they figure out which one is the best and stuck with that um so before testing You obviously need to set your measuring right, okay, so You'd be doing quantitative analysis which answers What is happening on the website and how something is happening on the website? So that would be analytics tools like Google analytics slash universal analytics case metrics mixpanel or whatever you like and The point is that in that in that situation you need to set your goals You need to set your up to set up your events You need set to set everything up that you would like to to track on the website and that's important for you And like I said You would be also doing behavior tracking that would mean that you would install a couple of scripts Like crazy egg click tail in specklet Which are all the tools which provides you with an information about where people click on the page So Google tells you okay I went from page a to page B or I spent that much time on page a or page B And this one tells you people were clicking on an image which didn't have a link and you say oh Damn, I forgot to put the link in there And you wouldn't realize that with Google analytics because it wouldn't show you the click on the element which doesn't have A link in it right so behavior tracking and the tools like heatmaps and session recording would show you those kind of Problems you possibly have with the user experience on your website Then you would do qualitative analysis which actually answers why you would answer you would ask people okay? Tell me what the problem is Did you possibly enter all your all of your credit card details, and it wouldn't go through Maybe it's a problem in any in a in a browser in a version of a browser and somebody couldn't pay even if they wanted and You didn't know that so you ask them and they say well You built a beautiful website, but I don't know where to click and I've seen that too many times So somebody invests like a couple of thousands of euros in some beautiful design and it's a spaceship But nobody knows what to do with it. So and they say okay Please some button or some indication on where should I click right or you just do something like polls Where you put you know those those small? Annoying things in the bottom right which Pops up and say okay. Are you satisfied or whatever? You need to define sample size now this is a very important point I Won't bother you with statistics even if you if it would be good if you are Past statistics 101, but let's just say that you need for a proper test You need to know what's your sample size you need to know how many people would you include in the testing? Otherwise, you're just Doing something random. You don't have the exact number about how how relevant the test is So you need to know by using the calculators Just remember that you need two-sided to test so not one sided right So You would actually be dividing the sample size based on your traffic volume on that page or the block of pages and You would define it with the current conversion rate and it would say to you you need to split test for the next 44 days and each of the variations needs to to include 1500 visitors for example, and it needs like 10 conversions per variation and that's your sample size and 44 days is how much you would be testing with slight variations, but I'll go back to that later So the only way to actually be statistically valid is to do it like this Seasonality now you can test for 44 days, but imagine in those 44 days. It was a Christmas But then you have a problem because people are behaving differently during Certain Holidays during certain seasons for example, you have a shop which sells eyeglasses or sunglasses for example and Nobody's buying them in September or October right so you need to know your business and you need to know What's the seasonality in your business in that case? you might need to prolong the period the period which during which you're testing or to Shorten it a little bit and to put in more juice so that it could go faster before Christmas sales or cyber Monday or whatever Then you need to all you need to align your split testing with business strategy What that means is that you wouldn't be split testing just the first thing that you think off and you said like oh Cool, I saw yesterday a webinar and they told me that it's really nice thing to test the button Call to action maybe test the title or everybody just test the image So the lady watch the lady is is turn on this side or on this side. Let's split test that. Okay, so if it doesn't align with your Business strategy and goals what you would like to accomplish and what you would like to measure and to improve Just don't randomly split test some some things So first determine keep KPIs that would actually mean key performance indicators, which is actually What is important for you? Which are the indicators of your improvement of the website and? Please do I'm not I wasn't I wasn't doing a due diligence in this one I didn't make testing roadmaps, but I realized that it's a very good thing to do Why because when you make a testing roadmap that means I want to test these 20 things in the next six months Why because if you listed all of the things that you need to test that you want to test You would be able to prior prioritize you would say actually okay this is more important at this point and I can leave this For let's say in three months something like that if you didn't do that you just be doing it randomly So saying okay, let's do this for example Then business cycles So for example, I have worked on a website that sells Swiss watches So one Swiss watch was like 15 20,000 euros Their sale cycle was between two and six months First time people came on the website until the point. They actually bought the watch Six six months right so imagine you would be split you were split testing for Maybe two months or something like that. You didn't think about the actual length of the business Business cycle and the waves they are in there some of you have have seen probably That there are waves in the people behavior online So on Monday everybody is ecstatic So the traffic goes like this and on Tuesday. It's about like this and on Friday. Everybody is Eager to go home and on Saturday Sunday if you're not a business which Works during whole week you have low amount of traffic. So take that as well into account So if you're beginning on Monday and ending on Friday, you'll be missing two days on Saturday or Sunday And you will be missing valid data For that to to to emphasize that so test minimum one purchase cycle or And or right to business cycles So technical check Today There were a lot of presentations about the SEO about the speed about the technical aspect in the background of the website so I'll be just saying that if For example, you have set your split test you have planned everything you have measured yours that you have done your homework and statistics You have done prepared everything But you forgot to test it on an iPad So sometimes People actually from the iPad might be taking like 20 or 30 percent of your traffic traffic amount And they're not seeing your test properly Imagine what that would do to your results and you would be changing the whole website according to Behavior on the desktop and you wouldn't know that something got really really messed up on iPad or a bit niche example Let's just say that recently I had a problem between version 39 and 40 of Google Chrome and It be it happened in the middle of my test When the test ended I realized that actually in the version 40 they just did something which messed up my test and So I had to do it again Because I don't have valid data because in the middle of a test something happened so Do do your homework? test in the browsers devices screen resolutions Check your server resources So you don't need to have you don't want to have a website which loads in like five ten fifteen seconds, right? turn off any plug-in that you can and especially Think about the the ones which are unnecessary, of course, so I'll I'll be sending you this on on the Twitter. I'll be posting this So you have a resource list of all the possible resources that you might need for the analytics behavior Tracking design analysis or feedbacks some of them I have mentioned previously You have as well the duration calculator Then previously mentioned side speed page page speed tools then ping them, right? So all of those are used for side speed measuring then browser compatibility that I've just talked about so browser lane cross browser testing Browser, etc. So you'd be testing actually one version of the test across browsers variations devices, etc statistical significance Resources and the duration calculator So now what to test? Okay, so we get back to that thing What to test is it the just first thing that we saw and Or possibly something which we think it's fancy to test right now Well, just align it with your goals. So I won't I don't want to repeat myself I have already told you that but I'm just emphasizing the importance of that just think about what your goals are and according to that just split as that And don't take the good case good case practices because there are no universal rules in split testing Let me just say it one more time. There are no universal rules. So green button doesn't work all the time orange button Also, so it's a point of actually you have you have a good case practices Amazon can tell you okay, we have split test on our huge visitor base Something for two months and we realized that green button kicks ass That's good. But maybe it's not for your website. So You can take good case case practices, but take it with a grain of salt. So you actually need To look for patterns and the way they're thinking because you probably don't have budgets like Amazon does right for split testing or Google and they're they're crazy idea of split testing the color, but You have the way of thinking of actually implementing the changes and thinking Okay, so buttons should be probably here or maybe image should be here But like I said take it with a grain of salt I've seen in many cases that one website Doesn't have to have the actual image of the product on the left and everything else on the right I've seen it just mirrored and conversion was just skyrocketing. So There's no universal rules Now how to test on WordPress Let's just say that you can You can split test page a versus page b So in that case you would be using one of the tools for split testing like visual website optimizer or optimizer or whatever And you would be making two different URLs and you would be sending traffic randomly to page a or page b That's the simplest one Then you have dynamic changes on the website in that case you need a developer probably if you have a dynamic website Which would make actually jQuery slash JavaScript code which would dynamically change the page And in that case you still have one page with dynamic changes of the whole page That's my my preference Then you have a multivariate test as we mentioned so you can dynamically change the elements within the page Then you can split test the whole page template or the whole website template By the way, have you noticed that there are no images in the presentation? Yon, okay. Yeah, there are no images. There are no jumping dogs or owls Funny cats or whatever I'm actually split testing this presentation because I've given something similar in Prague like month and a half ago and With all the funny elements. I had a room like this so Okay, you're not like that. Okay. All right. All right. Okay. Okay. So anyway Now the WordPress testing and this is the resource list for some of the things that I mentioned like testing providers like visual website optimizer optimizer, etc. Some of them have the WordPress plugins and Let's just say that for example My personal preference is content experiments within universal analytics first things first thing why because it's free and because I can do my own jQuery javascript and changes on the page and second thing because it plays much nicer with Old sorts of plugins. So you don't need a special plug-in for split testing or external service that would do that for you So you're doing it on page with job jQuery with Google Analytics Nailio is one of the examples where you would be using their plug-in for doing the The randomization and some plugins on the on the right then What are the general testing pitfalls and myths? Let's just say Let's just say that stopping test at 95 percent when it's reached is not a good idea first thing because you need To honor your sample size and your test length I've seen in many cases people saying okay So we want it to be 95 percent valid. We have a five percent error a five percent Possibility of error. So now that we've reached 95 percent. We're good. So I've seen it happening the happening after two hours so In beginning of the test probably if you have a lot of traffic one of the variations will jump up and We'll just make a big difference comparing to the other variations and you would reach 95 percent So no, that's not valid So you need really to honor what we've talked about So that you could have your split split test valid then Test results for one experiment. They're valid forever no again because Possibly you have been split testing. I know for example in in a season Which isn't your typical season or in the meantime you changed your ad words call to action Or for example in the meantime you did something else on the website you maybe you improve the speed and With the improved speed you would be having lower bounce rate more people will be staying there and you would be having a different result so as a rule of thumb, let's just say that once a year if you did some huge change on the website retest it because you might get different results and And one of the Let's say one of them the myths and one of the lies Test without conversion rate lift is failure. No, it's not Because from the failure you would be learning what didn't work and you wouldn't do it again, right? So it's not a failure because if you track it and and remember what you did wrong and all the other Methodology that you did like so you properly set up measuring you did your homework in the statistics, etc so if you did all of that and You didn't get good results. That's completely okay Lots of let's just say CRO gurus conversion rate optimization gurus would tell you online and That split testing is let's just say easy and it's a gameplay Well, let's just say that it's not that hard But the problem is that not every test wins Let's just say that about 30 to 40 percent of tests if you're experienced wins So I needed to brush up my skills for a couple of years to reach the point where 40 percent of my webs Of my tests win. So don't be discouraged if your first test just fails That's completely okay Then we're press website testing problems. We have talked about the caching Well, let's just say that in many times. I've been cursing caching developers. Why? Because often testing cannot run and doesn't play nice with caching plugins Especially for example, if you have You have a couple of versions of page for you developers How many of you actually develop the website online instead of local environment one Nice, okay Last time I asked this question a question. I got the full room of live ones Okay, so anyway Imagine that you are actually trying to see if something works and you raise the cache in the chrome and you're not seeing the new variation and Imagine on top of that you have a caching plug-in Then the then the fun begins because you cannot see on which of the variation changes that you did how they're performing and whether the browser is displaying them properly so either Try different caching plug-in or turn off the caching completely That's one of the the common pitfalls with the WordPress website If you don't have to use caching if you have a light light website in the split testing just don't use it Or sometimes just excluding certain scripts like for example Split testing script gets minified by caching plug-in right and It just messes up the display of the and the the the the way plug-in works So just exclude the script and it will probably work properly Then probably you might be having too much plugins like How many of you have the websites which have over 50 plugins on the website? I've seen that okay. Anyway Over 50 plugins that would mean actually a page load time of about 15 to 20 seconds and that's nice because the guy is really sure that His website is going well Because he has all of the whistles and bells that he read on online and he thinks that it's just fine And when I added on top of that my split testing It went on any unusable so 22 seconds. No So just turn off all of plugins that you don't need try to lower page load time As much as you can Testing plug-in is not playing nice with some other plugins. That's something which you will see too often Unfortunately, which is why I would actually recommend the version with jQuery and Slash JavaScript on page through Google Analytics because it's the let's say it's the cleanest version You don't need to put Plug-in for Google Analytics. You would just say okay. This is the code. These are the changes do the content experiment through experiments API and the Variation that you need would be pulled and the page properly displayed in many of the cases So or just try other testing services So these are some Different resources that I would I would recommend so some people to follow on Twitter some blogs to follow some possible books to read That let's just say that some of these actually influenced influenced me like don't make me think my way of actually thinking and and doing the design and Now about me. This is the part where also doesn't have an image. This is my mistake. So anyway, I Am a consultant independent consultant working in in the field of analytics and adwords and CRO for let's say a couple of a couple of years lots of websites and I've been working at manage WP until recently and now I'm Here talking about these things as an independent guy. So That's all from me for today and you're free to ask questions. Thank you Okay, we are we are taking one or two questions because we are a little late Any question? Man, okay, then I need to thank you very much. Right. Thank you very much Igor. Thank you for being here