 It's starting here. We have the demo over here. It's going to help us figure out which way you're going to do it. How's everyone doing today? Great. This is a really nice turnout for a Sunday afternoon. So, thank you so much for coming here. So, has anyone heard any of my talks before? A couple of people. So, my talks are famous for being controversial. So, who here is the designer of the room? Okay, who's a photographer? Okay. I'll probably be knowing this off all of you. So, just be aware of that. But this talk is not your traditional technical talk. It's not a technical talk. But first of all, who am I? What am I doing? Why am I standing up here? My name is Mike Demo. Hey, I'm Mike Demo. I'm an evangelist for Bold Grid. I'm also the Treasurer of Open Source Matters. Anyone here know what Open Source Matters is? That one person. This is the Joomla Foundation. Oh, how did he get in here? Call security. Here's the deal, guys. Tools are tools. I don't care what CMS you like. WordPress, Joomla, Joomple, et cetera. Use the tool you like. I use WordPress. I've made hundreds of WordPress sites. I love WordPress. I see who's Joomla. It's fine. It's okay. Guess what? Open Source is more important than what tool you prefer to use. Excuse me. I'm a publisher manager at the Joomla Will Conference. I'm also a husband, father, video game lover, and a former Disney cast member in the close target of the coast. I'm less impressed with that sound. I'm also the owner of Own and Full Dog. You can see up there. My name is Fulana. I think you can see some pictures of me during my WordPress evangelism. I work at KPU and some other events out there, including going to college, teaching people college kids about WordPress. This is a picture from the Snap conference. It's a city. I was one of only three men at this conference. It was a very interesting event. And those are professional male dancers dressed as unicorns. So, yes. I go to a lot of different types of events talking about WordPress and everything out there. I also love WordPress because of the diversity. I love how inclusive this community is. I love how we all get along and follow the world in the same direction. And I love how they can meet people from different religious backgrounds, ethnic backgrounds, political backgrounds, and other views. And I love how this community comes together. And even if we might not all agree about every single topic, we all are here for the love of WordPress and the love of open source. And that is why I'm here. Because WordPress is people. And WordPress is really the people's what drives this. This is an amazing event. I mean, this San Diego camp hasn't spent an amazing weekend. It's been great. If you are interested in Bold Grid we have a booth out there. You can talk to me out there. But this talk is not about Bold Grid. It's not really about open source. It's about ducks. Who here came because the talk had ducks in the title? You can be honest, it's okay. This talk is about ducks. And we're going to get to the ducks in a little bit and I promise. But this talk is about AB testing. But this is also a talk that is very interactive. I will throw things at you. You laughed and I'm joking. But I'm not. This is a talk where I'm going to ask the people to give me stories and ideas. I'm going to have a discussion. Because this is a topic that a lot of people think is really complicated and all this stuff. But it's really not. I have one major theme and you'll get it catching on a little bit. And how you can make money on AB testing. So anyone here have any experience in split testing or AB testing? Okay. Do you care to give an example of some of the such an experience you have with it? Sure. The last time I worked out we were in media and publishing company. And so our effort was taking articles as well as versus Cool. Overall do you think it was successful? Yes. Anyone here have an example of AB testing that they feel was maybe not as successful? Anyone here have a story to tell? Okay. That's excellent. Thank you so much. So basically here's what a lot of people do with AB testing. They don't know what they're doing. They just change stuff. They change more stuff, results go down and they have no idea why. Or they change six things at a time. Or they're just throwing stuff at the wall and see what sticks. But here's the thing. Is that AB testing needs to be controlled. It just needs to be controlled. And we do that a lot of different ways. So when you're doing your AB testing do you use any sort of tool like optimizably or BWO or another plugin? Yeah it is a product called ABKC. There's lots of tools out there. So it needs to be a controlled test as does any of this sound familiar where you do a lot of changes and then you see your sales go up and you don't know why. Or they go down and then you change a lot more stuff and then more results happen and you really don't know why. Because people when they do changes on their website they're pretty much just guessing. Who here when they launch a new project when you launch a website who here wins in writing the goal of that site? Okay, two. Those people that raise their hands is that goal binary. Will you be able to look a year later and say yes or no, did you meet that? Exactly. So a lot of times it's like oh well we want our website to be more or get better or do more sales. Here's the thing guys. Is you just setting yourself up for failure. You launch a website, the client loves it. They think you are a superstar. They are so happy on your wonderful site. And then six months goes by the honeymoon period is over. You ask for your additional work and they're like hey do you want to do this stuff? And they're like eh I don't know. The website's not doing as much as we thought or we hoped or we wanted. Does that sound familiar as they haven't had a similar situation that happened in the past? Because we set ourselves up for failure. A website and many online marketing that we framed is nothing more than a magic money machine. I literally used this exact terminology when I did this for Fortune 500 clients at downtown agencies in Minneapolis. I'm sorry about that. But it has a blizzard right now and my wife has snowed in and she has been removed for three days so I'm happy to be here. It's a magic money machine. That's all it is. And it's a magic money machine that we have to tune to make more money come out. And I ask a client hey if I can prove to you every dollar I stick in this magic money machine can stick out two dollars. How many dollars would it give you? Well they laugh and I say oh seriously and then they're like gold. I mean it's a smart goal. It's specific, measurable, attainable and time-specific. So that in a year from now we know yes or no if this website project was a success. It's all subjectivity out of it. Because guess what? It was the secret. Clients don't care about design. They don't. They care about their results. They might like how the site looks and wanted to look pretty and all that but at the end of the day every client has a purchase. Every website has a KPI. And that KPI might be page user engagement. It might be traditional sales. It might be stickiness on the site. It might be how it interacts with your media campaign or things like that. But if we don't write down these goals we're never going to know if they're successful or not. And that's where AV testing can help you with. Because if you can say at the beginning to a client this website is not going to be perfect but we're going to test it and adjust it and make it better and tune in this magic money machine to make your results better and we're going to hit our goals every single month and we can prove it. Then they're going to be paying you every single month. Residual income is amazing in AV testing and clients love to do it because we're doing nothing but increasing their results. We're just increasing their results. So it has to be controlled. With that, what I recommend is you use a tool like BWO or Optimizely or AV tasting without what you want to use. There's lots of WordPress plugins out there. You want plugins that are better than the other. Find a tool that works for you. Then it's going to do the results for you. It's going to make sure it's statistically relevant. Anyone here have a doctorate and applied data analysis? Wait. Do you guys don't have PhD in data science in your in the web field? Well, you should just quit right now. Why are you doing your job? Nobody knows the stuff that you need to know. Actually, I did this top one. I checked Riverside and he raised his hand. He's like, yep, Harvard. And I'm like, okay. Here's the mic. But here's the thing, guys. Did you know that websites when you do tests, there's margins of error? Do you know there's all these different things that can affect it? Here's an example. I had a client that was like, oh my God, my results are this much month over month. I'm like, great. You do? Well, I did all these changes, blah, blah, blah. Okay, great. How are they year over year? What? They were comparing September I'm sorry, October through November. They're in retail. Guess what happens in November? People buy stuff because it's the holiday season. And year over year, they were 20% down. But they don't look at the big picture. That is why schools like BWO, Optimized Lead and everything else out there will do the work for you. They'll tell you if the test you're doing is a winner or a loser or you need to keep testing. And the crazy stuff works sometimes. I'm going to talk about that in a little bit. Kind of a crazy different test. Do micro testing. So when you're doing your tests, we only change in usually one element at a time, but we're doing multivariate. We started out with just one element and we assembled it in the evening. So everyone here know the difference between micro-variant and single-variant testing? So here's the main difference. So let's say you have a button, right? And that button has two colors. We'll say red and green as the two different colors are your tests. If you are just testing the different colors, that would be single-variant because you're only testing the color of the button. But let's say you were testing a test that the button was green, but also a different font. There would be two variants on that one specific test. I highly recommend highly, highly, highly recommend the start with single-variant testing because multivariant testing can get very confusing very quick. And here's the truth. Very few of you in this room have enough traffic to sustain multivariant testing. I don't even touch multivariant testing in most cases unless the client is pushing at least 5,000 daily users. Quite honest. That's why single-variant testing allows you to focus on a single element. We're going to talk about what you can test in a little bit. So, what to test? So what are the different things you can test on a website? We talked about buttons and fonts. So what are other things? Same track. Play out. Great. Photos. I'm sorry. Call the actions. CTA. Copy. Copy writing photos. Great. It's all amazing ideas. And here's the thing that so many people sit down with. You're like, I don't know what to test. I don't know what to test. My website's perfect. I am an expert. I know what I'm doing. I did the best practices for the internet. And I know all the internet. So I don't need to test. Here's the result. Here's the dirty little secret in our industry. You guys ready? We're guessing. We are guessing. Every single one of us are guessing. Period. Now, you're not going to say that you're an expert teams and you're experienced in the user case and the case studies and the light papers. And all of the experiments you have is not valid. It is. It's extremely valid. But it's what gets you started. You need to then use that as a launch pad site. Because every set of users are completely different. And I guarantee you, I guarantee you, you do not know who is visiting that website as independently as you think you do. I'm going to talk about some crazy examples. A little bit later. And I definitely guarantee your clients don't know who is visiting their website. I had a client. It was a food company. They were convinced that it was college students that were buying the student product. They were convinced of it. Whole marketing strategy behind this. Well, guess what? No one was buying it. Not as much as they should have been. They were doing ad campaigns and so we started looking at it. You know who was visiting the website? Not colleges. Single family mothers. Because it was a natural food product. I had no added sugar. So guess what? We changed our strategy and we started doing some tests through the single family homes. And guess what? Sales went up. You know why they thought it was college students? Because the marketing manager just graduated from college six months ago. And his dad is the owner of the company and guess what he had in his dorm room? This product. And we all laughed at that but it happens all the time. It happens all the time where we think we know everything. And guess what? We don't. And testing is a way to get paid to make your client's website more better. It's good for your client. You owe it to your client's users. And you get paid for it. What's not the like? Anyone here familiar with HubSpot? Anyone here at HubSpot Agency? No? Okay. So HubSpot now it teaches their agencies to do a method called growth driven design. GDD. And growth driven design is a fancy word for testing. They launch a launch website and then they do monthly iterations. They start tests to make it better. Those agencies that do growth driven design make 86% more revenue than their agencies that don't. It's just A.B. testing with a fancy name. Luke Summerfield is actually a friend of mine. He's got your order. He's the director of education over at HubSpot. It's the same thing. But if you tell the client from the beginning that testing's a requirement for the project they're not going to fight you off. And you can approve the data that it's worthwhile to them. I see a lot of people that say, well I can't convince the client to do testing. You've been through a good job at the beginning of the sales cycle. You didn't set your expectations right. How many times has this happened with a client? You meet with a client and they say, last time I made a new website was four years ago. Ah, I jumped forever. Six months and we had all these meetings and it took all this time and it was just a really bad experience. Now we finally got the website done then we didn't touch it again. Until I called you and you're going to solve everything. We'll be another new website until I don't touch that again for four years. Everyone's in this two to four year website cycle. It's silly. It's a waste of money. And guess what? The clients don't like it any better. Sit down with a client and say, you'll never have to make a new website again. This would be the last website you're ever going to launch. Because you're going to do iterative testing every single month and make these adjustments based on data. Data driven decisions. That's all that matters is data. And the website is going to be dramatically different 24 months after you launch it. Think about what you want to do with the website as a launchpad cycle. So what's going to start the project, but it's not the end of the project. You set this expectation from the beginning and it's so much fun to test. You can have so much fun with the AB test and apply it by some fun examples. So what can you test? Binds. You can test so many things with the bind. Colors, fonts, styles, sides, drop shadow. So I once had a client, a national insurance company. Their KPI was quote request. Okay? Quote form for their insurance. We increased their insurance quote by 80%. I tested everything. I hired expert UI and UX experts out there. We had quote piece books. We had everything. We had copyright issues. Everything was performing badly. I tried a two-pixel drop shadow. Two pixels. 80% left. I didn't believe it. I ran another 50,000 users through the test. Same result. I tried it on a different page. Another 200,000 users through this test. Same result. A two-pixel drop shadow gave them 80% more quote request. Can anyone guess why? Why that would be the reason? Anyone have any ideas? Try to see the button. Look more like a button. Okay, impressive? What was the 80%? It's the reason. It doesn't matter. It doesn't. It doesn't. It doesn't matter. I'll give it junk about is the data. It is the data. I don't care about data. If you try to find reasoning why users use the internet, you're going to go crazy. Trust me. Trust me on this. You don't know what's in people's heads. Now, you don't want to look at it and look at general case studies and general user behaviors, and then try it again. There are things that most people do interact with, but every city user is a little bit different. That's why we talk about this talk. Which way does it get fixed? There was a case study. There was a picture of a duck and it was married. The left-facing duck converted 70% less than the right-facing duck. That's the question why. They say, well, because the beak is pointing to the call to action or the duck is forward looking and then every five talks and I have over 30 of these recorded online and you can watch them and see them yourself. We decided we'd lost them and happened. But it's because the duck is looking into the future. I don't know why and I don't care. It's because it's data. So we got text. We test so many things with text. The font, the size, the style, the case. Copywriting. Professional copywriting versus non-professional copywriting. Do you know there are sets of users that perform better with fun content than if you have a professional copywriter write something? It's true. It's really true. Thank me so much for writing all of their copy for them. They were so thankful they could watch their website. It was corporate if some. And this was a company that had millions and millions and millions of dollars on management. Everyone interacts with websites way differently than you think they do. So text. You can test so many different things with the text. We like to call it action when you said earlier. Images. You can test so many different things on images. Who here doesn't use the stock photos? Who here refuses to use stock photos? And if those of you who's a photographer? Here's the thing guys. Sometimes stock photos perform way better way better than original photography. And there are sets of users that perform really badly with stock photos. And you can get into all these case studies about education and all this stuff. But guess what? I test it. I just test it. If the results show that a stock photo performs better than an original photo, I'll use the stock photo. And guess what? Clients, even though they might say they want original photography, you prove you give them the money, you show them that their KPIs are working more, they will stop caring about originality real quick. Now, I'm not saying original photography is bad. There are lots and lots of cases where stock photos just perform really, really badly. But it's not a binary thing. This is not something you do 100% of the time otherwise. It's this way over the highway. And I know the right answer every single time. And for those of you that can't afford original photography, there are stock photo companies that specialize in non-stock photos. Which is kind of fun. You can test everything. You can test every single thing. And Tessie doesn't even have to be on the website. So, here's where you go a little political. I still talk about the presidential campaign and how they did every testing. Oh, yeah. This is the part where it gets dangerous. I did do this in Texas two days after the election, though. And I'm still here. So, well, Ty, I promise I'll talk about both parties. We'll talk about Donald Trump first. Two weeks before the election, the week before the election, one of the most successful fundraising campaigns the Donald Trump campaign ever did. They had to think, oh, I'm going to be on the Facebook page. Okay, fine. I might not personally want to get money that organization. But I get what they're doing. You own a piece of the campaign, but the Democrats do the same thing. Now, you were hired to do that. How would you technically solve that? You know, people want to donate money, and their name appears on the website, like the ticker, like Joe gave 20 bucks from Ohio or something. Or it'd be some technical ways you could solve it. Assuming you took the job, the video, the flash, the donation page. There's all these ways you can do that. Here's the thing. We can all agree that Donald Trump cares about his career. And for those of you that think these crazy ideas, I can't do it, because my time's pretty important, we can all agree that the president cares about his career. I think that's something that is just pretty safe. Here's what they did. A week before the election, we go based on the donation reports. We don't know exactly how much, but it seems to be one of the most successful fundraiser campaigns he ever did. People donate money live on the website, and it's shot to a live webcam laser printer. Printing out names 10 at a time. This is not a third-party group. This is not a political action committee. This is Donald J. Trump's official Facebook page. Everyone laughs at this. It says, oh, look, he's so disconnected. For those of you that don't think this was deliberate, you are solely, solely mistaken. Millions and millions and millions of dollars are raised. And guess what? Every political party hires companies to do their online fundraising. They get commission. I know, because they used to work for one. This was completely deliberate, because they knew who they were going after. They tested it. They did something very similar for the convention, but it was very, like, highly reduced. Get your name on the Megatron convention, and it was super polished. That also developed into not nearly as well as this and what we know. And this went on. I mean, before the election, I had this off of my computer for eight solid hours. I couldn't believe what I was seeing. Every once in a while, I had a handwritten note that said, here's how you're doing it. And you would see a disembodied ant come down from the screen. So, please put all those thoughts away in the minds to important. I've done this for eight. I've done crazy ideas for Fortune 500s. And the president, who is our current president, who we all can agree cares about his brand, right? Because he has hotels and chains and ties and everything. They acknowledge me that his fans supported him. He was able to do it in symbolic. I raised a lot of money. Hillary Clinton does similar things. One of the tests that she was doing a lot for the election is they would change the homepage, like how much you would default from donating when you click on it. One of the most successful ones. And they did different things, like book offers, like a copy of the book, signatures, and what was very successful for that one was donating money and maybe meet me for dinner and things like that, or go see Hamilton with me. And they did a lot of tests for that. Again, same companies, not necessarily the same company, but very similar companies, like pay commission calls. So, what is my crazy list? So, this is literally the A.V. testing list that I use, and I'll show you how to get a copy of the book I got this from at the end. Professional versus unprofessional. So, everyone's like, oh, no, I can't make unprofessional website. What about my reputation? Here's the thing. You can make an unprofessional website that still needs brand standards. Here's a way to give the caveat. Please, please, please pass all tests by your client before you make them live. Otherwise, it might not, it may not so well. Although, I have been guilty of just doing tests with an A.V. but I shouldn't have done that. But, professional unprofessional. There are times unprofessional websites work. There are times where no images on the page convert way better. As anyone ever got in one of those fun, amazing letters, those eight page ones that ask you to save the unicorns or something that have like scribbles in the margins. And I'm not saying it's not an important cause. I've personally never seen a unicorn, I think they need to be saved. But, there are certain people that those directly sponsored letters convert. And there are certain people that don't. They do convert less than they did maybe in the 80s. Probably. But they still send them out because they still convert. Same goes for the web. There's a reason why someone has a lane on a website and it's one of those directly sponsored landing pages that goes on for a million miles and has screenshots of someone's PayPal account. Look how much money I've made in 24 hours and the longer you scroll the better the offer gets. By the ski book, right now, I was just at the ClickFunnels conference which is a software that makes landing pages. They had like 250 people that made direct response pages in the last 12 months make over a million dollars in revenue. Just using their software doing this doing this model. So sometimes a professional does work. Photo versus illustration. New stock illustration. Higher as I know. Sometimes photos aren't always the best thing. One color versus another color. Test the colors. When you test the colors, test the accessible colors. Does anyone here know what WCAG 2.0 is? So it's an accessibility standard. In there, they have double A and triple A. They have contrast ratios that are mathematically set for accessibility. So when you test your colors, test your contrast ratios. Not only is it good for accessibility which one in the American adults suffers from something covered under that standard. It also is good for results because guess what? If people can see what you're off through is they usually opt into it more. It's amazing. It's really weird. Low brightness contrast versus high brightness contrast. Sometimes overflowing it out works. And I know the designers in the room are crazy, but here's the thing. It's not about you. We are really cocky in this industry. We like to play God. If you're wondering why Wix Weekly and Squarespace is doing so well, because people historically don't like dealing with web designers. They've been burned in the past. They feel scammed. They feel like they're not in their best interest. And I'm sure that's nobody in this room, but we all agree that there have been people in our industry that have done not the best for reputation. Why do you think do-it-yourself solutions are doing so well? So please, test it. And when you test it, don't test all the traffic. Just test five or ten percent. Use one of those tools and see how subset of this traffic is going to get these tests. And let the results win out. And if the results say yes, these tests are over the win success, test it again. We've got another page. If it then still wins, then great. You've found a winner. Incomprehensible. Border versus no border. We saw the SpinaToy. Monics versus clear image. Static versus animation. And if this was moving, you could see the card. Ugly versus not ugly. There are times where sometimes having really striking photos, one way or the other will give you better results and there's times that it won't. Especially in the last year, we've noticed the overproduced person is not converting nearly as well, especially with certain demographics. They want to see real people. Some of the things that's been in the news, we're trying to be more real, more natural, not overly sexualized or any of that sort of stuff. But every setting of users are different. Static versus interactive. Get 50% off, or you can choose the kids up or the daughter to get 50% off. Like for example, oh, but there's only two. Professional stock photo versus amateur informer photo. One lamp versus another lamp. See, get it started. Get it started. One call to action versus another call to action. Get me free trial today or click your internet access. Images versus text only. I know a lot of people really cringe at that. Oh, I can't have a text only thing. Yes, you can. Test it. Here's the most interesting thing is the fastest growing segment of the internet right now are on browsers that can't display images. The fastest growing segment of the internet are on devices that can't display images. We'll talk about the developing world. And for those of you who say, well, that's not my... Those people aren't going to my website. Well, think about what your website is. If you provide... Here's an example. If you provide data for victims of domestic abuse, maybe you should provide a text only option because maybe they only have a flip phone. We stop making assumptions and we start testing using the results. Upgrade versus angled. Plug-in versus Lingerline text. There's our friend the drop shadow or regular border versus a false foreign drop shadow. I love drop shadows. Custom add design versus familiar UI elements. Familiar UI elements is something you can implement. Have you ever wondered that Facebook now started doing the menu at the bottom of your phone because of the devices are getting so big. Now a lot of websites are mimicking the bottom menu look because the hamburger is physically too far away and it's very similar with that. I am not saying to lie and say your Facebook but you take some of those ideas. A visual image versus mirror image is our friend the duck again. One font size style case versus another font size style case, very easy to test. Standard add shape like a rectangle or a custom add shape like a circle or in today's internet standard add shape like a circle or a custom add shape like a rectangle. Positive versus negative Your spouse again, a sick fighting with your spouse and it can't even work sometimes. I once had a client they were a product that stopped stove fires. Guess what? You know what worked? Your mom will leave the stove unattended and will die because stove fires live enough for one cause.