 So my name's Demo, and you're probably wondering who is this guy? I'll get to that in a second. But first off, who came just because he had ducks in the title? A few people. I mean, I get it. I get it. So there will be some talks about ducks, and there'll be some talks about AB testing. This is not your normal talk. The slides are pretty bare. It's a discussion. So I encourage people to interrupt me, ask questions, yell at me, death threats, whatever you want to do. So the last time I gave this talk was actually over in Europe, so I'm having to edit all my nice political jokes, now that reality hit. So, but yeah, but actually the campaigns are actually a great source of AB testing because they did some very cool optimization techniques for their fundraising efforts on both sides. And I'll be mentioning some of those just to talk about it conceptually, how they targeted the markets. But first, a little bit about me. So I'm an evangelist at BoldGrid. Everyone's like, well, what's an evangelist? I'm like, it's like the revival religious thing, except I talk about web technologies instead of talking about God. So that's my full-time job is every week I'm at a different word camp, hosting a event or something like that for a BoldGrid, talking about the web and all the cool stuff out there. If you haven't yet gotten your BoldGrid shirt, please stop by a booth out there, take it because I do not want to take them home. So please grab them, they're organic cotton, we spare no expense. So I'm also the treasurer of Open Source Matters Inc. Does anyone know what Open Source Matters Inc. is? Hopefully they don't get thrown at, but it's actually the legal and financial body of the JUMLA CMS project. And a lot of people are like, why are you? That makes no sense. I'm an open source fan. I love open source technologies. I give back to all open source communities. That's why I am an open source evangelist, as you see in my logo up there. I think that you begin to learn things from every event. That's why JUMLA invited Matt Mulingwijk, who we all know and love, out to the JUMLA conference. And it's good to know the other tools and see what's out there. And I just fost and free an open source software. It's good for everybody. Whatever tool you use is great and WordPress is obviously my day job. And I love WordPress, but I got my start 10 years ago in JUMLA. So I try to get back to that community too. Now, I'm also the sponsorship manager of a couple of conferences. And I am a former Disney cast member, father and two, a little fur baby. And of course, a husband to my wife who sees me about once a week, maybe two times a week if she's lucky. It works out really well. Tipped to stay married, just travel all the time. It makes coming home a lot happier. But yeah, we can talk about A.B. testing. So first of all, when I talk about A.B. testing, I talk about control tests. So it needs to be controlled. What I mean by that is a lot of people, they don't really know how to start this A.B. testing thing. And when I first of all, what is A.B. testing? A lot of people don't know. They know it's a way to help you make more money. I'm a realist and everything I do is based on data and we'll talk about that. And all of this is from very real examples of agencies I used to work at. I used to be the digital manager of a pretty large advertising agency in downtown Minneapolis with clients such as 3M. They used to have an entire digital department for a company that did hundreds and hundreds of banking websites. And generally these are all just tried and true examples and ways to think about kind of A.B. testing. Has anyone here have done any A.B. testing before yet? Okay. Keep those hands up. You're not done yet? Okay. So does someone want to volunteer, want to tell me what that test was and what you were testing and what the results were? Sure. For a health supplement retailer on the site, he wasn't getting enough checkouts that he wanted. So my proposed test was he wasn't building up enough trust for the check out. He had a ton of Facebook fans. So when we put a Facebook like box that showed the greater fans by the final check out button and we tested it and increased sales by 70%. Okay. So I'm just, when you guys talk, I'm going to repeat it back not because I'm deaf, but so that the camera can actually hear it. So you had a client who had a supplement site. The conversion wasn't as hot as you may be able to like it. You had a lot of social proof. In this case, Facebook likes and you decided to put a Facebook like button on the checkout process. Okay. So yeah. So you would see, okay. So if I was checking out for the first time, I would see all the social proof that all these people were liking it and then had improved the conversions by 17%. Excellent. One more example that someone would want to share. Sure. A lot of it is social media that we do or a marketing agency. And the first thing is sizes. You know, one size is not enough at all. So we have to work with that. But it's always the message, the reach, you know, based on what platform. And then we decide where to spend the dollars. Okay. So again, they do a lot of the market agency, digital marketing. And they do little pilot tests. Have I heard you correctly? See which one is most successful for that specific campaign? Because not one size does not fit all for every client at a campaign. And then decide where to kind of put that dollar on there. That's a good, that's a good point. Because I like to say that A.B. testing in websites are a magic money machine. And our job as developers is to make the magic money machines work as well for our clients. And I get in trouble a little bit with these talks. Because a lot of developers are like, some of the stuff you said and some of those examples that you gave, which you'll see in a minute, it was just, they were just ugly. And I'm like, yeah, but the client made 300% more money. Do you think the client gave a shit? No. It's our job to do our expertise and knowledge and the best things we can to make our client's project successful. Every client has different goals. Maybe their goal is more conversions or more revenue, like we heard from the supplement site. Maybe it's just more likes or visits or more stickiness on it. But whatever that is, we should make sure that we try to do the best for that. And with that, I believe A.B. testing is one of the best things you can do. And so when you guys do A.B. testing, who's who's done this sort of A.B. test? Okay. I have a website. I made a bunch of changes and then the website made more money. And I don't know why. Yeah. So that's usually what happens. You go in and you make a bunch of changes and you're like, okay, I changed these 16 things and this happened, but I have no idea why. So first of all, A.B. testing needs to be controlled. Make sure, you know, you need to test one element at a time. You can do a multivariant testing and we'll talk about that in a little bit. But unless you have a lot of traffic, it's going to be really hard to do multivariant testing. So I always like to do one change, one thing at a time, one tagline, one color, one font, whatever the case is, and test it side by side. I use a lot of different tools. There are some great WordPress plugins. I like BWO as a paid subscription for A.B. testing. There's other ones out there like Optimize. Lee or whatever. And you want to use a tool, whatever you do, use a tool so you can actually see if it's statistically relevant. And that's the other thing that I learned the hard way is that people, they get excited by movement, but they don't really know what any of that movement means. I had a client that was getting thousands and thousands of visits a day. And they're like, oh my God, we did this one thing and I made $5,000 in one day. And I'm like, OK, but if you look at compared to A.B. testing as we tested multiple things, it actually wasn't statistically relevant. It was within the margin of error. So you need to make sure that you know what you're doing because I can't tell who's had something like where your client asks you to do something, like update a website or and then the site broke and then they blamed you for it. Like, you know, like, well, hey, you did it. You're the last person to touch my site. Well, correlation is not causation. And I think that we as people just try to be like, well, this thing happened and then this thing happened. And obviously they're linked because I want them to be linked. So I'm just going to say that they're linked. But you need to make sure they're statistically relevant. And I did this for an insurance company, you know, we did a bunch of different testing and they were it was a insurance dental insurance company. And we did all these tests. We were testing tag lines and pictures and testimonials and things. And we were able to move in their number, their needle by 80 percent. More quote submissions. By just changing the border radius of the fricking button. I mean, what the heck? I had psychologically done testimonials and I had all of this beautiful graphics and all of this stuff and we ended up just changing the border radius at the after doing 80 separate tests and the clients happy because they make millions. They're making millions more money, millions more dollars on just a little change. And we like to just over complicate things as designers and developers. I find we love to do all of this cool stuff. But at the end of the day, just test the small shit and which, by the way, I swear in my talk, so it just heads up on that. I try not to, but it happens. I'll also give you a checklist that we'll talk about. So statistically relevant, micro testing, that's kind of why I said, test little things at a time. Don't test multivariant testing. Just test a couple of different things one at a time. Test a color, test a tagline, test a picture, things like that. And there's some really good proof to all of this. That's kind of going out there. So can someone give me some ideas on what things you can test? Just shout them out. Border radius. What could what could would that do? I mean, seriously, look nicer. Border radius, other things you could test the color of the call to action. Cool. OK, button labels, text on the button, the placement, OK, of the of the button. We went from a two-page checkout to a one-page checkout. OK, simplifying the checkout process, reducing number of steps, things like that. OK, what font weight, OK, cool. Yeah, and we're going to talk about all that. Excuse me. First of all, we're going to talk about buttons. So, yeah, all these things you can do about buttons, you can test the button position, the button color, the button font, if you have an icon on the button. Something that's worked really well for me in the past is having multiple lines of text in the button, like a main button called action with like a subtest, like sign up, you know, sign up for three and then a little subtext, you know, we won't sell your information or whatever the case may be, depending on what you are doing the thing of. So, but in sizing colors, if you have icons, the call to action, all these different things, and you can very easily do this with some of these tools, depending on how much time we have, I may actually jump into VWO and do some live testing on someone's site. That's a cool thing about VWO is you can type in any domain and test. It just doesn't go live unless the JavaScript's on there. So I've like done this on cnn.com and whitehouse.com and things like that. It's kind of fun, but if we don't have time, I can definitely sit down with you over in the happiness bar, which they would like to let you know, speakers after their talks will be in the happiness bar if you'd like to talk to us. So, text, text is a big one, you know, testimonials, calls to actions, how much care, how much care do you have, you know, is it friendly, is it first person, is it third person, all of these different things. Text is a very big thing that a lot of people overlook. Images, and we'll talk about that in more detail when we get to the checklist. Images are funny. Stock images, who uses stock images? Raise your hand. Stock images. Who refuses to use stock images? One person. So it's kind of funny. Stock images are like, you know, love and hated by the community. You're like, I never use stock images because, well, they're stock images, or I only use stock images because the client has no money and whatever the case is. So images are kind of funny because I've tested stock versus not stock. I've tested ugly versus sexy. All these different things and images are really those things that actually kind of work. And it will kind of beat you up inside, like when we talk about some of the examples, some of these things as designers, and it beats me up inside, but I test it. I do have to stress that when you have a client, please make sure you approve all tests with them before you start doing them, otherwise they might get mad at you. And please make sure that you explain why. You say, we're just going to test a small sample. We're going to test, I personally test, 5% of all traffic to my client's site. If they have enough traffic to support it, that's the problem, right? Is that if your client doesn't have a high traffic site and you only test 5% of the traffic, it might take four months to get enough statistically relevant traffic to actually make a decision. So you need to kind of decide, I don't like going over 25% out of any circumstances because I don't like everybody seeing my A-B testing and I don't want to have the A-B test being permanent until I get the statistically relevant data. So images, definitely look at the images and everything basically has to all point to this. It's usually at this time they talk about ducks. So HubSpot did a study. Believers HubSpot have to relook up the way paper, but they do so many, it's hard to keep track. They did a study. They had two landing pages. One had a picture of a duck facing to the left. One had a picture of a duck facing to the right. And the right facing duck converted three times more than the left facing duck. Can anyone guess why the duck facing to the right converted more? Okay, we read left to right. Okay, if you're looking to the right, look to the future. Okay, so if it's to the left of the page just like the duck is saying the text, okay. Did the right facing duck face the CTA? Did it face a paragraph copy, not the CTA, but... It should face the CTA. Okay, I've heard, oh, yes. Something to do with right handed more than left handed people. Okay, right handed versus more than left handed people. I'll get answers. I've asked this question all over the world and I get all these different answers. And you know what I tell them? What the real, you know, what I think of it? I don't care. I don't, and neither should you. You need to stop trying to think of why things happen because it'll drive you crazy in A-B testing. Have the data drive your decisions. If your data is statistically relevant and you have enough of a test to make it statistically relevant and it's working, by all means, just keep pressing the button for the magic money machine and keep taking your client's money and being everyone's happy. If you're trying to think of all these psychological reasons, you're going to drive yourself crazy. And yes, I know there's lots of people out there with psychology degrees that do very expensive testing and consulting and eye pupil things and all this expensive stuff. At the end of the day, just do whatever moves the needle. If it's working for your goal at that site, because every site has a goal. It might be revenue, it might be staying on there, it might be liking the Facebook button. It's something. Every project has a goal and it's our job to try to help that client realize that goal. And if you do a good job and you can prove that, you know, you're making the magic money machine spit out more money on the other side, you know, you know how many clients have said, okay, Mr. Client, you know, I'm really sorry we only made, we only increased your profit by 35% this month instead of the 14 it was last month. Would you like to go again and try to get 35% again like we have the last six months? So do you want to cancel? Nobody's ever told me they want to cancel. And that's the point is we want to service our clients, but we also need to make a living. So my checklist, my actual checklist, I took this from a book that took it from another website. It's sourced up here. And I'll show you how to get this book at the end. And I'll show you how to get it without spending money. And this is my actual checklist and it's funny. Like it says, don't laugh till you try it. So professional versus unprofessional. And yes, I've done this. I've used Comic Sans and AB tests. But guess what? Sometimes that sells. And this is where I talk about one, the dangerous possible campaign thing. So one of the candidates who ended up being successful, he did his fundraiser a week before the election. He said, hey, donate and your name will be on my Facebook page. Okay, it's kind of a cool social way to do it. Donate, your name will be on the page. Can anyone guess? And if you know it, don't say it. Can anyone guess how he did that? Like what are the technical methods that if you said, hey, you know, I would like to have a thing. So when people donate their name flashes on my page, can anyone think of any technical ways that you could achieve that? Or the website if you don't want to do Facebook page. JavaScript, live video editing software where it comes in with a nice fade. There's lots of polished ways. You can make that look nice and polished and professional and dare say presidential. But you know what the Trump campaign did? I couldn't believe it. It was on my screen all day. I couldn't believe this is what major political candidate with supposedly all the money in the world was doing. And it raised more money than any other fundraising effort he did. He had a live webcam of a laser printer printing out the names. For 14 hours, 20 names on a sheet. And not only that, they had a very high tech method to get you to donate. Every once in every about 20 minutes, a hand would pull down with a sign. A hand written sign. That said, go to this website. Donate blah, blah, blah. Hashtag drainage, drainage sample, whatever it is. And it worked. And I was watching the numbers. He was having hundreds of thousands of views all day on this so low tech example. But guess what? He probably would not have made anywhere close to that if it was a high production polished value because he knew his target market. Yeah. Now, it's funny to think about. But if you think about it, think about the target market of the Trump campaign. Think about who we connected with. A lot of it was middle America. Lou Coller. And it actually resonated with them. He tried this once before, not nearly as successful. It was before the RNC. You could donate and you would have your name on the big jumbotron during the RNC. And that was produced in value with all this production and lens flare. And all this after effect stuff because they paid $800 for the software. So we got to use every feature. And it raised not nearly as much based on the numbers. They don't release the exact campaign success, but based on what I saw when witnessing it. So yeah, it's very interesting. So photo versus illustration, pretty easy. Test of photo versus illustration. There's lots of great stock illustrations out there. One of my favorite sites is actually Fiverr for just AB testing illustrations. If you're an artist or know an artist that can do it custom, great. Please do, but not everyone does. So Fiverr, you can get a custom illustration of five bucks. It's not perfect, but it's enough for a test. One color versus another color. Pretty easy test. Low brightness versus contrast. High brightness versus contrast. Border versus no border where it blends in with the site background. Clear image versus blurry image. The blurry image stuff is actually pretty easy to do and worked quite successfully. Static versus animation. Ugly versus sex appeal. And yeah, I've put ugly people in that ugly on site before. And it works sometimes. Sometimes it does. And it's just, it's funny. Like we all wonder in today's day and age, because anyone landed on one of those DR or direct response long form pages that just go on and on forever. And then like have the testimonials and the screenshots of the actual PayPal account with all the money that I actually made with this tool. There's a reason why those exist. It's because they work for a certain person. And, you know, they don't keep cropping up out of nowhere. They work for a reason. So static versus interactive. The Clinton campaign was really good about this one actually. Their donation page. The home page had like these buddings you could choose to donate on the home page like $5, $10, $25, whatever. It wasn't actually a button. It was just a picture of a button. And no matter what you click, it like defaulted to like $100 or something. But it's just the idea. It's the idea of choice. And I guarantee you, they do AP testing. One of the best things you can try if you want to look at some really good A.B. testing live is go to puckovin.com. It's Wolfgang Pucks, like pressure oven thing. And do it for like different IPs. You'll see so many different layouts. And it's just interesting how they, and the price will change like a $100 swing. So it's pretty interesting how they do that. Professional stock photo versus amateur informal photo. We all can take amateur informal photos pretty easily. We all have these really expensive cameras in our pockets. So definitely hang out with that. One layout versus another layout. Another again, pretty easy to do. One called action versus another called action. Try different things. The dental insurance example, hiding things like, they used to say a 30 second quote. And we would say, we tried no obligation quote. And we tried, get an instant quote. We tried all of these different things. Get a quote, have, starting at $9.99 or whatever the rate plan was. And of course the border radius is what did it. Not better. See what happens when you fixate on why stuff happens? You never get over it. Images versus text only. You might think text only is crazy. Why would I want to do text only? But again, it depends who your target market is and doesn't hurt to test. And if it works, it works. Okay. Butting versus blue underline text. I've done this quite a bit. Works pretty well, depends on it. Regular border versus a false border in drop shadow because, why not? A custom ad design with familiar UI elements. I like doing mock Facebook layouts a lot, quite a bit simply because it's familiar and people like to know how to interact with it. Now it's at this time in the talk everyone says, okay demo, that's your real name. Which may not seem democular, sort of a demo. There's too many mics in this world. I'm trying to help everybody so just don't think about it too much. But yeah, they're like, this might be fine for some back county website. But I have a big client like CNN or something. And remember, I've done work for national brands. I've done work for 3M. And all this works, you just have to make sure you follow their brand book and their brand guidelines. All these concepts can be done with any brand under the sun. You just need to massage it towards that direction and of course get approval ahead of time. Yes, they have their colors and their fonts and their logos and their layouts or whatever. But you can have it look nice, but still be unprofessional-ish in terms of how this goes. So original image versus the mirror image. It's our old duck thing going over again. I accidentally answered that. One font size style case versus another font size style or case. Standard ad shape versus a custom ad shape with CSS. It's very easy to do 100% border radius. Not that kind of fun to do. Although, someone looked at the web recently, I think a square is the custom ad shape now and the circle is like the standard shape. I see it everywhere. Positive versus negative. This is always fun. So in this example, it says want to fall in love with your spouse again or sick of fighting with your spouse. I had a client that it was a device kind of like life alert. You know, fall in and it can't get out, blah blah blah, a device like that. Do you notice they sell those things in two different methods? They sell them because the client isn't the senior citizen living independent. It's the kids. That's the client. That's who they're targeting with these ads. So one example is they'll say, do you want the peace of mind that your mom and dad can live independent and not have to go to an assisted facility and all this wonderful stuff, all these nice wonderful feelings. That's one way to do it. But you also see ads that say your parents will die. It's going to be your fault. I mean, you've seen these ads. I'm not making this stuff up. You see it out there. There's a reason why this stuff works. I did this with a device that did, that stopped stoves. It was a device that if you walked away from your stove, it shut off because stoves are the number one cause of house fires. One example, it's like, you know, would be like, you know, for like a lot of college kids, we were selling it to other parents for the college kids or whatever. But be like, oh yeah, you know, peace of mind, safety, blah, blah, blah. But we also tried things like, you know, we reported, you know, we had like statistics this week. There's been this many stove fires that burned down houses because of unintended stove fires, blah, blah, you know, get this product, whatever the case is. So definitely try it again. Approve anything by your client first. Generic versus relevant to time period. You probably see a lot of these ads, especially like for like auto loans or whatever, like, want a girlfriend? You can meet single women here or home alone on a Friday night when it happens to be Friday. How did they know? Or, you know, like single women in Wichita looking for you. I live in Wichita. How? Oh my goodness. It's my lucky day. There are single women in Wichita. Or want a girlfriend for Christmas, assuming it's around the holiday time. Mention prices and discounts. Regular $20 to 90% off. Or, you know, leave it a mystery. Good for your feet. Add to cart to see final price. There's two reasons why company retailers do that. One, sometimes they're obligated to based on the MSRP requirements that they can't tell you the price until you're in the checkout process. But a lot of times it's just they want to get you to add it to your cart. So, because there's all sorts of data around that. It's amazing. No trust logos versus trust logos. Ascena, ABC, NBC, CNN. Trust logos can butt you in the butt though. You need to be careful with that. Has anyone here heard of my pillow? The pillow? Okay. Has anyone heard? I live in the Trans City. So, he's like a local rock star in there. Like, so silly. I have the pillow personally, full disclosure. I like it, but the guy's silly. Anyway, they would always say, better business bureau A plus rated for years and years. Anyone hear what happened with the better business bureau with him? You heard about it? Yeah. So, the better business bureau took him from an A plus to an F overnight. And now he's like being like all surprised and blah, blah, blah. There's lots of reasons why and has nothing to do with the product. It had to do with the sales practices or whatever, but and not honoring the code of ethics of that organization, which you sign when you pay the membership fee, whatever. Anyway, but now it's hurting him because he's been using this as his sales crush this whole time. And now he can't. So, now he has to find another sales method. Now, I personally love the pillows. I think they're great. I think the sales are really frustrating because I never know when they're on sale or not, which was the main complaint to the better business bureau. So, whatever the case is. That being said, you can also use your weapons of influence. These are from influence by Robert Kennedy. And I, of course, have just pronounced that. You look at scarcity, you know, offer ends at midnight. Why? So, wall supplies, last authority. More doctors recommend this five out of six dentists agree. Consistency, current visitors who take small steps, asking for a major commitment, you know, get all those little guesses, whatever the case is. A lot of, like, digital content providers will do this to get it down with a free e-book and then, like, do little steps, like, you know, along the way to try to get to the big yes of the big sale, liking using photos of real people versus models, similar to what we heard from our supplement example earlier. Lucy is a new mother, just like you. She uses our product to keep her baby happy. Of course, that product is obviously an iPad. Social proof. I don't have kids, by the way, if you can't tell. So, I'm never home. I mean, we may have kids, but they might not be nine, but yeah. Social proof, one million satisfied customers since 2010. Testimonials, I lost 56 pounds in this program. I didn't, obviously, but that example did. We're giving you a free trial, or my favorite thing, a no-risk trial, but they charge you, but they still refund you later. I like that term, no-risk trial. But again, you can test all this. It's all about avi-testing. The point of this is to give examples. I want to make sure we have time for questions. Give examples of ways you can use this in your business. And we'll ask questions at the end. Appeal to the eight universal desires. Survival in German to life. Life extension, you know, if you can do that. Enjoyment of food and beverages. Freedom from fear, pain and danger. Sexual companionship is also something you can use for selling. Comfortable living conditions to be superior. Warming, keeping up with the Joneses. Caring protections of love, ones or social approval. Fear and pain. If he died tomorrow, who would take care of your family? Get a free life insurance quote today. Because nothing takes care of your family more than cold hard cash. Benefits, not features, three times warmer than other brands. Maybe with Maverick Gustav, ask a question. Be controversial. Use short words. Because short words are easier to read. Like Trump or Clinton. Vote for the census. Five senses. Sight, hearing, touch, smell, taste. Example of using touch. Talk about a tell a story. People buy, I used to sell TimeShare years ago by the way, which might be why I'm jaded. People buy in a motion to justify with logic. Seriously, and it works on the web the same way. And again, it's our job to try to paint that picture to help our client move the needle and make the magic money machine work. Because if you can tell a client, and I do this with clients, if I have a machine, and I put $3 in a machine and $5 came out the other end, how many dollars would you give me to put in that machine? Well, the answer is on the limited dollars. And that's great for us, because those mean billable hours. So, plain have fun. That's the thing I want to make sure that you remember is that A.B. testing is some of the most fun you can have for the clients money that you'll ever do. And you definitely want to make sure you take advantage of that, while also ensuring your loans are being later. So, I'm going to show you how to get a cool book about A.B. testing. The slides are on slide sharing linked on the website. Now, the reason I have these steps outlined is because although I like this company's products, if you don't follow these steps exactly, you'll spend anywhere between like $30,000 to like $30,000. So, it's an exaggeration, but he does have some expensive products. So, you can go to .comsecretslabs.com slash free dash book. I need to zoom in, that is awful. Okay, you'll see this, and you'll see this form. Fill out the form. Easy. You will see an offer for something. If you want to, by all means, buy it, but just decline the little checkbox, which will be automatically checked. Then you'll see a video with an upgrade offer, which you say no thanks, I'm good at the bottom of this. Now, I want to say I've paid for a lot of his products in some of his clubs. For a while, I was a member of his monthly newsletter subscription for like $75 a month. I don't know the exact price. I like his stuff, but his stuff is geared very heavily down the direct response industry, as far as like his paid stuff, because he sells like funnel informed building software for people that are selling health supplements and just A.B. testing and digital products. But you can use his philosophies. He's one of the largest experts out there kind of teaching this stuff to just everyday people for any client out there. He was actually featured on The Profit, the CNBC show. He helped make a site for the Neth Watch episode. So anyway, I like his stuff, but not everyone likes checkout processes like this, which is why sometimes companies adapt like GoDaddy. You say no thanks at the bottom. There'll be another offer. You say no thanks. You'll see this page. Close it. If you click anything, you would have bought something. You get an email and then unsubscribe from the email only if you don't want beyond this newsletter, which will offer you more services. Again, I like his stuff. It's just not everyone wants that sort of information. So I've got about 10 minutes for questions. So I'm not going to be able to do a live demo, but if you have questions, I want me to look at your site with ideas. Please see me in the happiness bar. I'm happy to sit down and talk with you. I'll talk with you longer if you tell me you have picked up a bold good shirt ahead of time, simply because I don't want to take them home. I'm kidding. I'll talk to you just as long if you even if you don't get a bold good shirt. But so questions, who has questions, comments, the threats, anything? Yeah? What's the reasonable sample size or at least the minimal sample size I try to get 5000 users through my tests personally. Everyone has different numbers, but what I like about paid tools like VWO is that I'll tell you when the ratio is enough to be statistically relevant and actually give you a recommendation like kill the test, extend the test or implement the variable or whatever the case. I try to hit 5000. That's my personal rule, but I'm usually dealing with sites that have a lot of traffic. Then let's let a lot of brands and sites out there. So other questions? Really? No. I have another one real quick. Sure. You said the other thing works better. Yeah. Anything else that seems to be Yeah. It kind of depends on like how like the social consensus is going. You also kind of kind of look at like who the target market is and are they educated or are they not? What the persona is is a great tool. Makemypersona.com which helps you walk through how to make a persona. It's a free tool powered by that helps by offers. And you kind of look at it because if they're educated or uneducated, have kids not have kids whatever the case may be. It might definitely can definitely change what you might want to do. Other things that I've seen that work well are a lot of like micro steps like get a free ebook to try to opt into something or whatever the case. I'm always a fan of trying to make it statistically relevant. There's a great tool out there that they can call JL Mailer. It is an email newsletter system that will personalize the website based on the person that you send the email to. So you can send the email to Joe and Joe clicks the link. It's going to, it'll say Joe's name in the text content. I've had that work really well. I've also had a mad lib form. So instead of a form it says hi my name is form field and I'm interested in drop down. So it's kind of you know more conversational. Other questions or things that you've experienced? For the email or? It's a service. It's called VWL. Victor, visual website optimizer. They have a free tier I think or a low cost tier and it works with any CMS out there. There's some things on WordPress out there on the repo. I don't use those as much because I don't like changing my database permanently. And unless it's a permanent change. So I don't want to have like two pages and have people go to actually two separate pager post types to try to test and try to like deal with different URLs and things like that. I want every element to be the same except that single change. Which is why a JavaScript solution like optimized via VWO kind of works for that situation. I played with it. What I like about VWO is you don't need to know any code to do any of the changes. You can literally click a button and drag it across the screen and it will do the change for you even if you don't know how to do the code change. Now if it works, you'll have to figure out how to do it on the final product. But it's a very easy drag and drop editor. Other comments, questions. We've got a couple more minutes. Three. No? Yeah. What's your greatest discovery that I've often like? I guess my greatest discovery was to stop trying to figure stuff out and stop caring. But seriously, I've learned to embrace data and do database decisions. I learned that the small stuff makes the biggest impact. I've done the big huge site we do and not knowing why stuff happens. I mean, Victor, over there, we used to serve on the G1 project together and they did a huge update to their equivalent of the plugin directory and traffic went down, like 80% less or something, way less. And it was a huge redo and maybe that wasn't the best thing for those users. Maybe it should have been small tweaks. But, and I also learned to test along the way and that has to do with kind of accessibility and things like that too, that accessibility is like becoming kind of a passion of mine because one in six adults has a disability that is covered under WCAG 2.0. WCAG 2.0 is a new ADA accessibility requirement that's going to be a Department of Justice mandated 2018, they've announced. There's already been lawsuits going out and have been lost and settled by companies like Target and like 12 of my clients have been sued after this new requirement. The Department of Justice is saying you have to follow WCAG 2.0 AA or AAA. It's a very complicated accessibility law. Please make sure you're looking to it because it's going to become standard. If you would like to make a lot of money become certified and have a test of accessibility because it's very hard and very expensive to do so. But I like to test things that are related to accessibility like color contrast ratios and things to make so that people with certain types of color blindness can still read the text and stuff like that. So, yeah. Could you think that the psychology side that makes sense when you have low traffic and low budget and you have to pick small number of tests and you can't do all the variables even if you do them individually? Sure. You still have to... But you don't have enough traffic, yeah. Yeah, definitely. I mean, if you need to look at what who's coming to your site and make the best decision for that site. So, yeah, definitely prioritize what you think might work the best. For example, in my insurance company example we wanted to test having... We thought, okay, maybe if people knew what the average price was that might help things like that. So, yeah, definitely look at... You know, try to look at what might make sense to your users but if you can, test as much as possible. But if you have a low traffic site even doing one at a time it might take years to go through that list. So, and what I like about VWO2 is that there's all these cool ideas. All these like AB testing ideas that they give you and show you like how long they are to implement things like that. And just because a test you read a case study and I worked good for one person it doesn't mean it's going to work well for you. Every user base is different. Anyone else? One once? Going twice? Cool. Well, that's exactly time. So, thanks again. I really appreciate coming here. My treat I handed was MP Mike. Thanks.