 Namaste. I've wanted to come to India all my life. This is my bucket list, and so this is my first trip, and it's been awesome Thank you. I will tell you, Muja Hindi Ati Hei Yeah, and those are the only four words I know But it worked with a couple of people at work that got really scared when I thought I knew what they were talking about. I Didn't know a few curse words which I will not share with you today. I hear they're really good ones So let's let's start today with a with a few things a Little bit about me if you're on Twitter Please please follow me. I am narcissistic Do enjoy followers. I'll follow you back I'll put you on a testing list, and then you'll have other Thousands of other people who are in on the Twitter in the Twitter circuit out there for testing I need to update this slide. I've done this talk a few times, and I'm at 25 years Now so I'm a quarter away there for that century of a career But I've started my career in development. I was a developer thought I was going to do that all of my life In sixth grade I said I'll be a developer. I love this. I'm gonna do this went to college got a degree got into development Then I joined the PMO Anybody work with the PMO Know me either I couldn't take it. It wasn't for me. I was too technical I wanted to be a technical person and so I got out of the PMO after a year Company I was with anyone knows Lowe's home improvement in in the United States. I know it's loving the back. Yeah But I was there for 20 years become a development manager and then I fell in love with testing I went to a testing conference, and I was a developer at this testing conference And I really felt out of place, but also realized I love testing and I love everything about it And it was around 2005 and I said this is what I'm going to do for the rest of my career NIT for sure I've kind of had experience of doing a lot of things in testing The different roles that I've had through the years have really given me a lot of experience and functional testing automation performance service virtualization and the last two jobs that I've had Just building up and starting organizations the current one I got to meet my team here who is in Bangalore and Four folks that are working with me. I don't usually get that opportunity to meet the folks that are working with me across the ocean So but it was great to meet the team Just want to give you a recap of what we're going to do today I'm not going to talk about automation even though we're at a selenium conference But I know when I talk to the organizers. This is something I think you're really going to enjoy today We're gonna have a lot of fun. This is gonna be your fun start to the morning We're gonna talk about no matter how good you are How good you believe you are as testers You have to realize there's a possibility of being so familiar with a product that our eyes Did not notice the changes and And we'll we'll talk about more of that today Tips to recognizing patterns and gaps that many visual testing activities miss and then lastly Techniques that you can use to become a better visual tester And when we talk about visual testing there's companies that do visual testing tools We'll talk about those briefly today, but then we also and and this is more about Using your vision to find what you need to do as a tester more than actual tools or actually the term visual testing that you've heard In your day-to-day work So tell me this how many times have you gotten in your car? You've drove to work or you've drove somewhere and you get there Or you come from work and you go all the way home, and you don't really remember the drive anybody do that Anybody go all the way through a whole drive, and you're like, I don't even remember what I did I don't remember where I was I remember I made it. I made it from point A to B. I didn't have a wreck Nobody hit me. I didn't hit anyone. I was paying attention, but I wasn't paying attention really paying attention And I found myself a lot of times getting all the way home And someone say to me did you notice that this happened on the side of the road or this is happening at that building or they're building something here And I'm like I didn't notice that because I'm zoned in I'm either using my phone over over a Bluetooth Or I'm listening to something on on the radio, and I'm just I'm kind of I know I'm driving. It's a routine It's kind of habitual and we do that as testers We get into the situation where we're doing the same thing over and over and we get so used to it We stop paying attention to things that are really there in front of us So who can tell me what this says? You know, can you read it? Does it say Mike Liles is jumping the conclusions? Does it? See your brain filled in those gaps You thought you knew what it was because you saw the top half of it and we do that a lot of times And we we fall into that situation as testers I'll start my clock late but we fall into that situation as testers and We and we see things and we fill the gaps and sometimes it's a good thing And he helps you to be a really good exploratory tester and sometimes it's a bad thing because you feel the gaps incorrectly What did that say? New York in the spring. How many things it says New York in the spring? Okay. How many does not think it said that? No Yeah There's two these Which your mind skips it because it takes that redundancy on the overlap and it doesn't notice that you're you're missing the You're missing that extra word and we do it But I saw this on a TV show and they were doing this in New York Which I never understand why that it had to be in New York I mean, you don't have to live in New York to read New York in the spring Can you tell me what is wrong in this picture the moon, okay? Two moons. Yes People try to figure out, you know, what is going on here to shadows, you know or something But yeah, he's on the moon and there's the moon. So it's a it's a problem Lots of rumors that this never really happened anyway, but this is definitely not a true story It's all true. We did never go to the moon. We never went back. So, you know, maybe true What do you see here you see a duck What do you see a rabbit? What do you see both? I should have done that. What was it yanny and anybody do the yanny? I I should have used that today. There's the there was yanny and there was some other anybody know what I'm talking about It's like the craze. Yeah There was a if you listen to this this recording Then you would hear yanny and then the next time you would listen to it Just the frequency of the level of volume you heard you'd hear another word So, Loro. Yes, Loro and yanny. Anybody know what I'm talking about? Okay, it's great. If you Google Loro and L O R E L and L A U R E L and Y A and Y you'll find the the sound clip and if you listen to it You'll hear them both and you'll think I don't I don't hear but one but you'll go back a few minutes later You'll hear the other and then sometimes you hear them both if you're weird like me What do you see here? Is this the good guy or the bad guy? Is there a mean face or a happy face? mean Happy we're at least somewhat pleasant. Maybe not too happy Now squint your eyes everybody squint I meant to take a picture when everybody does the squint your eyes really close and see what you see now If you squint if you squint where you can barely see it this one will go to normal and this one will go Anybody see that everybody so it's weird. I use this as an example of Sometimes it's if you have to step back and take a different look at what you're testing and what you're doing with your Your test plan to figure out how to see things that maybe you didn't see when you looked at it the first time or looked at it directly So let me tell you a little bit about myself. Okay, seriously. That's a joke And I'm not that narcissistic that I'm showing you this again, but did anybody notice something in my profile? My name isn't My name isn't my class I've done this presentation a few times and I've had one person in the whole and the whole all the times I've done it that's stopping me on the first slide and say wait, wait, wait, when does that say isn't yeah Usually don't catch that but good for you guys. So context is king. Definitely How many people work in a shop that does context-driven testing? No One person in the whole room. Oh, okay. Okay Anyone know what I'm talking about when I say rapid software testing? No, okay. Well anybody know James Bach see me after this presentation. I'm gonna I'm gonna help you change your world And if you're just against it, but I don't think you will be because I've been fortunate to work in shops That do requirements based testing and you take the requirements and you write the test cases and you execute those steps You complete the work, but in a context-driven shop It's more about understanding the app the application becoming familiar what is the context of what we're testing and My testers love it here in Bangalore because they can They don't have to write test cases now They realized after they got on board with me that there's still a lot of work to do because they do mind maps Anybody do mind maps in the organization? Love my maps look for X mind net if you're not using them. It's free to download X mind net And it's beautiful And it'll change your world if you can do your requirements and and look at your systems by doing mind maps And that's what I focus on in In a context-driven world is that is what we use is kind of defining how we're going to do testing Is by doing the mind maps and then and then taking it down at another level and become a very familiar with the application What that does for our testers when you're doing context-driven testing is it makes them more visual testers? They start seeing things, you know, if you take a requirement, here's your team requirements, and I write my test scripts They're focused there are these 10 steps and I'm going to do these things But if you if you look at context-driven I'm trying to figure out everything I can about the application and I don't really care with what the new changes were Or what the requirements I want to I want to make sure the requirements were met and the product is right But I want to make sure that the overall enterprise-level product that I'm delivering to the client is exactly what what they wanted or the stakeholders Now John Lubbock says what we see depends mainly on what we look for If you've ever said I I'm looking for a new car And I would like to have a red car and this model of the red car How many have done this and you go out and you say I want that car And then all of a sudden everywhere you look you see that red car or that model of that car And he's like I didn't see those before but now that I'm paying attention to this one type of car that I want Or this one bike that I want or this one bicycle that I want then all of a sudden now you see them everywhere And I try to do that when I'm trying to buy a new car because I want to make sure mine is special You know, I don't have that one car when people see it. That's him. I know who that is I don't always get that lucky. I usually end up with a car that everybody has And what are you looking for here? What do you see? Shadows good job. See you're learning by the end of this. You're gonna be teaching this class But a lot of people don't notice there's these are not camels They are camels, but what you see what you think this camel is right here along the Aerial view a top view of shadow and don't always trust what you see Anybody ever had this situation where you put salt on something you thought of a sugar you ever put salt in coffee Makes it even worse Make that mistake you only do it once But you need to be able to Make sure you don't always trust what you see Salt always salt will look like sugar so you taste it then you can tell once you get further deep into the details Then you can tell notice here Sometimes we get so narrow in our vision and what we're doing we see We see things that are not really there Or we see the wrong thing until we step back and be able to see the whole picture Some of the words today That we're gonna use today. We're gonna talk about is inattentional blindness This goes back to when you when you leave For home or you leave for work and you don't pay attention to things That's inattentional blindness and you you're you're not really blind of course But you're not catching things that just right in front of your face anamorphic art By stable images choice blindness and talk about neuroplasticity You didn't know you're gonna have a medical course today, right? And then change blindness as well before we really start talking about visual testing and vision We got to start where it all begins and that is with the brain and The human brain the one thing I thought was interesting when you start looking at the brain and how the brain works The part that actually does vision is in the very back furthest away from your eyes I don't know Why that was designed while we were designed that way and why that happened But it does seem the furthest away has to go through a lot of your eyes are up here and what does your actual vision is in the back One thing with the brain is just a few minutes ago We saw that if you tell your brain it has comparisons to make Decisions to make it'll start filling those gaps every single time if you look at it When you saw half my name my clouds is jumping to conclusions and you said I think that's what that means Because your brain was already filling in those gaps and trying to decide What what needs to be there because I can't see this some of your brain will take over and do that for you And a lot of times your brain will sacrifice facts over efficiency just pure Facts once you once that you believe that this is the truth You'll you'll go with that and we'll take mental shortcuts. We do it all the time Same thing with with how we test sometimes. We'll say I think I know how this should work So I'm gonna do this without really investigating maybe some of the things you should be doing To to make the make sure the product is ready to go Now can you read this? It's pretty easy, right? Yeah You know all the way through it. I highlighted strange because it was the one that messed me. I usually Couldn't get my I don't know why Maybe I'm strange. I don't know But you can go through this whole thing and the only thing That researchers have found is as long as you have the first letter in the last letter Long as there's some context to the story or the word you can fill in the gas your brain does the work I bet you didn't even know you could do this, right Now you can go home and say I don't really have to read real words. I can guess figure out a whole paragraph 60 70 people one thing is for sure is the brain is a survival organ It's designed us to solve problems related to surviving in unstable environments And even though we may not be the most strongest animals in the world Our brains are definitely the most developed We know that I've had some dogs that I think think as smart as I do They've tried to outsmart me for sure the one I have right now. There's a really good job of Outsmarting me or we go off on a tangent and tell you the story I lucked him out of my house the other day because I was working and I won't tell the reasons I lucked him out You can ask me afterwards, but he was Carrying up my house and so I put him in my backyard and I just let him out there and I was working at home And a few minutes later. I heard something at my door He had figured out a way to jump up pull the door handle and come in the house. He's never done that before and So now I have to lock the door when I send him outside That animals do get pretty smart So what what age do you think that the brain stops growing? Because we're growing, you know, we get to a certain size I'm hoping that both my brain and my Everything stops growing as much as it has been through the years But what what age do you think the brain stops growing? Never okay We're close to fully developed Probably in the 20s, but you're right. It never stops growing. Yeah, you can exercise It's the same reason we go to the gym every every, you know, some of us go to the gym Every day or every week and spend time there because if you keep exercising it keeps you fit If you keep your brain exercised, then you stay fit there as well Every brain is wired differently. We'll go through a couple of things on the brain here It's it was it was believed that they had that the brain slow growth at 20 and stopped growing at 40 But that's that's been proven not true Agree as well and what you do in learning life as you as you go through life You'll learn as you go and it kind of builds a lot of things. I've learned I Remember Steve Jobs saying we connect the dots Backwards not forwards. We know things that that happened to us We don't realize those things are happening and and then fate kind of helps us Drive what we're doing same things with our brains. We grow along the way What we learn today may help us be better tomorrow and the next day and the next week Kids you see different kids if you go to school, you'll see some tall kids short kids You know big kids small kids same thing with brains. They're brains developed the same way So don't don't tell your family members your brains better unless you believe that Okay Now here's the thing I think that a lot of people are surprised about and is that we avoid boredom by multi-tasking Anybody multi-task here every day that work. I try it I try to work on slides sometimes and watch TV and end up watching TV and I can't I can't do that Your brain is incapable There's research that's been done that there's like 2% of the people in the whole planet That are called supertaskers and they can do it and they can do it very efficiently I would say out of this room. Maybe one of you You know who you are you can do really good multitasking anybody? Okay But it's most of us Probably not the case probably 98% to 100% of this room. We're not very good if we try to multitask We try to juggle emails phone Tablets social networks And your error rate proven goes goes down I mean goes up 50% when you're trying to do multiple things at the same time as testers That's a challenge for us because when we have to do multiple things at the same time We we kind of get in that situation where We might have some errors in the way that we're testing because we're trying to test with coverage three or four things at the same time Speaking of boring things. Can you tell me? What's so unusual about this paragraph? No It is really hard to find so when you see it You won't be able to not see it. What is that? No comments. No, it's just because somebody Type it up bad No guesses and when I tell you it's gonna be so easily visible in front of you There's no ease in this whole paragraph now. I bet you can take it Where last line, okay, maybe it's X's What do you see ease no E E See you almost made me doubt my own presentation That's it here No ease there's no X's either We don't usually look for X's and paragraphs, right? Right, right There's just unusual Anybody see it was just afraid to say you saw no You didn't see Yeah, anybody seen the color card changing trick You have We'll play this for you Hi, I'm Richard. This is Sarah Are we going to perform the amazing color-changing card trick with this blue-backed deck of cards? Now the idea is very simple I'm just gonna spread the cards in front of Sarah and ask her to push any card towards the camera This card here, okay Now Sarah could have selected any card at all from the deck But she selected the card which is now face down on the table What I'm going to ask her to do is show us which card she selected I chose was in fact The three of diamonds, okay, excellent choice that card goes back into the deck Now I'm just going to spread the cards face up on the table a little click of the fingers And you'll see that Sarah's card here has now got a blue back Not particularly surprising what's slightly more surprising is all of the other cards have got red backs and That is the amazing color-changing card trick But did you see what happened? They changed the deck. How did he do that? Slider fans. I love magic with with cards, but now pay close. What's that? Yes, you're seeing it anybody else see her dress change No, good. I Know what? This is not about the cards Notice this now watch what they do when I'm Richard This is Sarah are we going to perform the amazing color-changing card trick with this blue-backed deck of cards? Now the idea is very simple I'm just going to spread the cards in front of Sarah and ask her to push any card towards the camera Okay Now Sarah could have selected any card at all from the deck But she selected the card which is now face down on the table What I'm going to ask her to do is show us which card she selected The three of diamonds, okay an excellent choice that card goes back into the deck Now I'm just going to spread the cards face up on the table do a little click of the fingers and You'll see that Sarah's card here has now got a blue back not particularly surprising What's slightly more surprising is all of the other cards have got red backs And that is the amazing color-changing card trick So Did you notice there was a gorilla in this picture that okay, so now you're starting to pay attention I don't know what they use the gorilla for I think they just wanted to say at the end of this Oh, by the way, we also got your game with this grill Now I'm gonna play another one for you which is very similar But I bet you're getting really better now because now you know the game right you know what's happening to you Clearly somebody in this room murdered lord smite who that precisely 3 34 this afternoon Was a brutally bludgeoned to death with a blunt instrument I want each of you to tell me your whereabouts That precisely the time that this dastardly deed took place I was hollering the brass in the master bedroom I was buttering his lordship scones below stairs, sir But I was planting my petunias in the potting shed Constable arrest Lady smithe But how did you know madame as any horticulturist will tell you one does not plant petunias until may is out take her away It's just a matter of observation The real question is how observant were you 21 changes Did you catch them? Clearly somebody in this room murdered lord smite who had precisely 3 34 this afternoon Was a brutally bludgeoned to death with a blunt instrument I want each of you to tell me your whereabouts and precisely the time that this dastardly deed took place I was hollering the brass in the master bedroom I was buttering his lordship's scones below stairs, sir But I was planting my petunias in the potting shed Constable arrest Lady smithe So Interesting there are several of these out there that you can find on youtube or wherever you go Anyone ever walk through a corn maze? You have Corn I'm an idiot. So I don't know if india has like corn fields and do you have corn fields and a lot of Have corn fields, but you've never gone through a corn maze anyone go through corn maze It's terrifying a little bit because i'm claustrophobic You know if i'm in an elevator and the door is set there This one here does this to me it gets to the ground floor and it sets there for me. I'm like, please just open I'm gonna I'm gonna start kicking the wall But corn mazes do that to me as well. I get a little claustrophobic because you get inside of a corn maze And it's great because if you've done if you've done mazes when you were a kid and you you mapped them out It was You know you looked at things like this and you see the aerial view It's easy to tell how I can get through this corn maze But my kids love to do corn mazes. I've taken to them to a couple and the last one I went to They gave they gave us a map they said you won't get lost It is huge maze. It was probably as complicated as this one And you start in I mean these are buildings here giant buildings So you come in here and then you try to make it to the end And you come out the other end and so my son and I we're going through and we're reading the map We're following the map. We know exactly where we're okay I can tell I'm here so I'm going to turn left and turn right And we got through this thing in just like 20 minutes And then we realized it wasn't that exciting when we didn't explore the rest of you know We had all these we basically went like right up through here and we were done, you know, and we're like, hey, we did it I don't know. I think we thought we were going to get a trophy at the end of this Oh where they were going to say Congratulations, you did it in 20 minutes. Nobody's ever done it that fast But when we got through we said, okay, let's back up And now let's backtrack and try to explore the whole thing. So maybe we spend another hour My daughter wasn't as happy about that but we spent an hour going through everything with my son and Here's kind of the view you've never been in the corn maze So you see this from the top and this looks pretty easy. Hey, I know I know which way I can go But when you're down in the corn, this is what you see. So you get to the end here It's kind of hard to see kind of the view of the way the map should look because you're walking in the middle of this corn That's taller than you are so my what it what it tells me when when I use this corn corn maze scenario is that You know using the map takes away your creativity I wanted the map. I wanted to have that backup so that we did get lost We can kind of figure out where we were even though they really didn't tell you where you were along the way so it's kind of like You would have had to figure it out anyway And you might get through quickly But you don't you don't get to explore the whole field a lot of times we test quickly In that situation where someone says I need this done in three days. Please get it back to me. Let's get it out the door We've developed it forever. Now we'd like you to test it for two days And and and you get in that situation and and we try to do it fast And there's there's creative ways to be fast to make sure you get the high-risk things and the high probability of failure Tested but it's always good to have more time in your testing plan as well And in the last you may miss some interesting important parts of the maze You just try to find the fastest path out I challenge my testers on my team To be the one that finds that one bug that makes the the project team go How in the world did you find this you know, how did you how did you find this problem? And they do take that as a challenge every time we test something it's always Nobody would have you know, lots of times our developers will say Nobody's ever going to do that and we're like what doesn't matter if we're going to do that or not We were able to do it and able to break it Or find it broken we didn't break it we start for relevance It is a proven fact studies have shown that We got to do something every 10 minutes to reset our attention So in this 90 minutes, I've got to keep you at least entertained every nine Nine times today. So We'll try to do that. So every every 10 minutes We'll wake you back up If I lose you But it is important to stay relevant and what I what I try to tell what this story is that when you're When you do start for relevance and you miss out after 10 minutes and and if you don't get that 10 minute Awakening when you're testing a lot of times we get into that Inattentional blindness and we start missing things that That we That we should be seeing but we're getting such into a routine that we miss it Just and later on you look back. Oh, I should have seen that I should have caught that Do you know the stressed brains? Don't learn the same way Lots of times of being stressed Our brains are wired to deal with with stress Lasting less than or equal to 30 seconds Anything more than that we start unwaring ourselves. So I'm not sure if I can handle even 30 seconds. I think but You may feel the same There's an old saying where the whether the lion eats you or you get away the stress lasts only a few minutes But what's what things stress or distract you at work? What things keep you from being productive in what you do every day? as a tester You can tell me I won't tell your boss My folks usually tell me it's you you're the one that stresses us because you tell us do this do this There are things that stress stress us out This slide bothers a lot of people has over the years Because of the hallucination and the amnesia people fear fear that this is really true What am I going to do when my work gets really really tough things get really really Difficult and stressful am I going to start having hallucinations? No, am I going to have amnesia? I think I have amnesia sometimes when I'm working Not real Now we have a lot of people here, but I want to do one thing Anybody play perfection anybody know that game? No, yeah, how many know that game? Just a few all right Now usually I've done this I've done this as a workshop and we have a smaller group But I want to do this today I just need two volunteers They have two volunteers that Braves to to play a game that my 10 year old daughter plays really well anyone Okay, who was it you and who else? First to stand up come on up. Yes, come on. All right now This is the game Show it to the camera. I'm on tv and I act differently when I'm on tv Use my right side of my face All right, so we will We'll do this because I don't know if everyone will be able to see it I wish I had was able to put this up. We can't put this up on the screen. Can we We can't put this up on the screen from there and it's okay. I didn't tell you that before Yeah, and we really don't we don't have to All right, and you folks can probably stand and see we'll we'll put it on this table Okay, so who wants to go first? You go first. Okay, you have to leave Take you go No, you just step outside the door do not listen to anything. I'll come get you. Okay, you're gonna be the second contestant You're gonna be first You folks can come around. I know it'll be a big crowd We're gonna see what's your name Sedita, okay You're gonna be the first contestant on perfection Yeah Okay, now I'm gonna tell you My 10 year old is really good at this Now I will tell you this thing Throws these everywhere when it gets to 60 seconds. You have 60 seconds to get these into this And my daughter does it in about 40 Okay And she's 10 she's 10 Okay, go You're halfway there 15 seconds This is not going well Okay Everybody everybody give a big hand big hand Turn me off Just for a second. So we're gonna bring me out. I need I need like a couple volunteers who'd like to play games on people So practical joke, okay Like four or five of you. Okay, I want you to come in and start I want you to be talking getting any space, you know, don't don't hit him Don't don't push him Yeah, somebody turn on somebody turn on your phone and play music, you know What I'm gonna do today, okay What was his name you can tell me on there Okay, what is your name? Raj Raj Ladies and gentlemen Raj Okay, right So the object of this game is for you to You got 60 seconds To get all of these into this, okay, okay She did it in 45 Okay, my daughter can do it in 40. She's 10 Okay Go Oh, wait, push it down, push it down, push it down That is a bug with this thing It's something wrong with Hold on. Hold on. I'm gonna give you one more chance Don't push the blue. There is a defense sending this back All right go the game is screwing up my my concept here. See he's been through this Halfway there this is a bug keep going It is breaking. What have you done to this thing? This worked on my machine I'm helping you now because I That's not the right place, but you are running out of time I've got two of these at home. I should have bought the other one I'll give you a few seconds A few more. You're only way further than she was Yeah, look at this nice See she knows how to do this because she's done it already. I'm so sorry Raj This is it final one Oh All right, give him a big hand. He did his best Okay, we just want to do this the rest of the day Okay And go home and buy it you can get this uh on ebay in some stores But I realized here that this thing I think traveling with this thing I've broken the the latch it should not come up like that Sorry I introduced new defects to your process Yeah, yeah, no it was kind of cool that it worked that way but no That was not intentional So So what does this tell us now? I've done this before and I had and I always have to wait to see Because I've done this before and the second person comes in and you tell them that they did it in 40 seconds I had a pretty good feeling that that raj may have been able to do it if it hadn't been jumping up and down on him And he would have been able to do it just because he thought somebody else could do it made him do it and and When when you tried it first You had no idea if anybody else had done it this way before so a lot of times working together helping other people figure out Here's what I did here's how I Solved the problem learning what other people have done In in testing or or even development kind of helps us out A lot of times in our in our test reports that we do in my Company We always want to know what are the existing problems that we've already faced So we can look for those things again if we've had problems in the past with an application Some people will perform better if they think someone else perform well And distractions will slow down some of us But what I find is some people handle distractions really well And they and you can be right in their face. I've had people put music in people's face I'm gonna write in their face. You're gonna get this you're gonna get this and some people get in the zone And they do really well there Now we talk about vision this this whole course today is about visual testing, but There's other senses that we should look at as well Reading hearing Seeing only and you combine hearing and seeing it increases. We're going to talk about that in a moment But smell is usually effective as unusually effective at provoking memories And I think about my grandmother who's been gone for Almost 20 years now, but I remember I can smell An apple pie and think of my grandmother just the moment I smell it I think about her apple pies and then our minds take us back to things lots of times We have centuries, you know It brings back things through our minds when we smell certain things or hear certain things Vision trumps all other senses. Sorry that I'm American talking about trumps anything. So That goes over really well in America every time I say that But children's books have a lot of pictures in them and We retain memory by hearing things 10 percent, but if you add pictures And you put pictures to a to a book, you know a lot of times you notice children's books Usually don't have a whole lot of words. They have a lot of pictures and kids remember the story Even though they may not even be able to read it But it increases by 65 when you have pictures included I need two more volunteers victims Everybody raise your hand. This is going to be really fun. This is a lot easier than that give me two all right you and But where oh, yes, come on up. Which of you is a better reader? Are you a good reader? How fast can you read? Come on up. Come on up. We'll put you on camera Welcome to youtube. You're a really good reader. How many how many books did you read this year? Oh, that sounds about like me How about you? What's your name? Just lean nice to meet you and Mohan Mohan and just lean give him a big hand How many books have you read? Okay, no books. Okay. I'm gonna give him the easier one All right, since you've read at least one All right, you come over here You stay here. Don't look at him You get this you get this I'm gonna give you 30 seconds to read this story No No, no, no because he doesn't want to hear what you're saying read it all now. I'm gonna give you a test afterwards I've got 10 questions to ask you Okay, remember what you're reading the intensity is building You're serious. You guys haven't read a lot lately, right? Come on, man. It's only like two paragraphs. Okay I'm joking Ready I'm letting just lean decide when we're done and he says he's done You're done. Okay Take the paper you done Okay Thank you. It's a competition All right, come closer get on tv Together smile Okay All right, here we go one of you hold this now You get to decide who who answers the question What was the boy's name you raise your hand or take your vote, okay Don cheese. Did you know this? Okay How old is he raise your hand first one to raise your hand? 13 Where did he go to school? He went on a green bus. Yes, you know what school was? Did you know green bus? Yes, okay, so he's got green bus. He's got one you got three. Okay, who knows what three cities he lived in Okay, yes, you knew this you remembered this Raise faster than him man. Come on Okay, uh, what did he see first when he went out? And it was and it was waiting it was on the bus. Okay. No, that wasn't what he saw first. Don't give him the answers for the next question What did he see first a yellow? Yes, you didn't see a yellow unicorn Seriously He's killing you. He is killing you now. All right. Okay, then what did he see next after that? big Tractor what color was that tractor? Right, right. What color was the tractor you forgot to? You know, you're killing me man What was driving the tractor? Wolf okay, so you both need a wolf. Okay. Now. What was on the trailer? There were multiple things on the trailer Grand piano. Did you know grand piano? Okay, it's okay not to remember you had the harder or I think Oh, okay. So you didn't read the other ones. You need to see it again. No, okay All right, so I would say he beat you on this Uh, you that one book in two years really helped you out because you read a lot better more detailed But give them both a big hand. I really appreciate you both for volunteering and I'll explain why I put you on the spot So thank you. Thank you. So now Jeslyn was at Um, jasmine. Okay. Sorry. Let me kill you jasmine when they did He was has a little bit of an advantage because This is what his his his His opposite the other guy. What's your name? mohan mohan Was at a disadvantage because this is what he read. Okay And jasmine he saw this Okay, so he saw all the pictures. He remembered the pictures. He remembered what he was reading So it kind of gave him an advantage to remember Uh, I tried this for the first time put the 13 in a big letter because neither people usually get 13 by just having 13 in the story and and it worked. Thank you the test passed But there's a lot to be learned from this A lot of times you can have a lot of times It's easier as a tester to be able to give give back to the organization exactly what you saw Trying to explain to a developer What you saw without showing them screenshots and reproducing that for them Sometimes doesn't give them the advantage they need to To go and find the solution My testers a lot of times want to help the developers which developers don't want their help sometimes, you know Ever been in that situation? I don't need your help. Just stay back But you know where natural born explores my 10-year-old daughter says to me dad I'm never going to be a tester now. She wears every shirt that I take home Um the sauce lab shirts and all the other shirts that I take home. She wears them all the time The t-shirts, but she says dad. I'm never going to be a tester and I said Unfortunately, you started out as a tester as a baby trying to put everything in your mouth That you could touch whether it was food or not food Jumping off the back of our couch at two years old And and she explored and my son was a little less wild as my daughter And he's older than her but but babies learn by exploring You know sometimes in Difficult situations as parents if you've been in that situation as a parent, you know that babies love to explore And that's why we have the childproof door handles and gates and everything else to keep them from going and doing things We don't want them to do But I think we need to If you've got a a kid and they want to ask questions let them ask all the questions they want Mine love to play the white game in the back of my car dad. Why are we doing this? Well, because we need to do this today Why then you ask the question then why and they just keep asking why until you reach back and please stop asking me so many questions But good testers testers learn how to experiment like babies I did I did a I did a workshop in california a usa Two months ago called testing like a kid We played games. We did things. I had videos of kids that have tested things I had young children Doing things and testing things and then I had people from the class do the same thing and what I found was We sometimes get ruined by growing up Sometimes and just as with many things in life. We allow Our what we learn along the way in life There's good there's good things you can learn and get better and then there's things that you kind of learn to become bad habits and Kids don't usually have barriers when they're test. They make great exploratory testers And anytime I want to test an app for my company a lot of times I can try with my kids and they they find things even I can't find just because they try things you don't even think of Including breaking the iPad which literally breaking it which is not very cost effective Another video for you. Let's try this one. See what you get Think fast check your head You have 10 seconds to answer Do you know 87 keep pause if you're good. Yes, it is 87. It's turned upside down um Everybody get it is 87 No I'll take you back to this really quick. I'm sorry you can see It's 87 because Think back The time something's good to see you anyone's seeing the cartoon where the testers on one side the developers on the number There's a six on the table and the developer's going. I see nine and the tester says I see six sometimes you got it You've got to define those different views that people have Now which way's this bus going Ah, that's a good question See, this is the first time I've done this in India, which is totally a different answer than I that I got in America the last time I did this You know where that's going kids will get this every single time another situation of not being ruined by Growing up, you know, which way it's going for sure Right It's parked That's I'm gonna use that. That's a great answer No, uh, it depends on where you are kids will tell you because they know kids know That the door opens to the curb so if I'm in the u.s the the bus is going to the Left because we're on the right side of the road And the door is going to open on the right side. So this bus is going this way But if you're here You you drive on the left side of the road And the bus is going to be on the left side and the door is going to be on the left Correct It could be reversing. Yes, that could be going backwards. That's another good idea Two good things. I'm adding that I'll get your names afterwards Nobody's ever said that great job Design of everyday things a lot of things are designed to be the same way I mean you go into your you go into your hotel room today as soon as you go in there. There's there's lights, right? I I learned in austria how to handle Using the lights in india. I was I was experienced after I spoke in austria last year But not being a world traveler until last year I didn't know I spent I spent about 20 minutes in a hotel in austria At a conference I was speaking at trying to figure out how to get the lights to come on and then I realized you Put your card in the in the door. We don't do that in america. You know, we just go in and put the switch But even then the switches are right at the door and so Most of the time things are designed that way things are designed the right way And I used to use the example of a controller controllers were always going to have the The power at the top left And then I was at a conference doing this very presentation and this is the controller I had and the power was down here so But I think they did it on purpose because the order button for ordering movies was up here now It's like hey if we change the design it helps us and now he's ordering stuff without knowing it Let me see gas pumps like this Gas pumps with debit yes or no credit and usually see an arrow here to one of these Buttons on the on the gas pump And I usually see that so you'll say there'll be a yes here with dots and then a no here with dots And I was at this gas pump. This is a real life picture right off my phone Where there was nothing so as a tester I pushed off all eight of them, you know, I start I start on the one I thought would be No, which usually over here is the way they usually do them in the u.s Probably here too But then I just started going through all of them and none of them worked and then I realized below that Was the touchpad that says yes and no nobody told me Nobody told me Change blindness Just every time I always think there's no way I can talk for an hour and a half on something And then I'm realizing now I got a hurry to get you guys through this today If you heard of change blindness We see things that slightly change and we don't realize it So when you an interesting way to do it a lot of people that look for change blindness Is they look at a picture and then you flash back and right back and forth Then you can see what changed in the picture. This one is pretty evident But there's a lot of tools And a lot of ways that you can test to find out what has changed from one one scenario one picture through the other I'm gonna try to be fast with this because we got about 30 minutes left And I got about 72 This video shows a participant from 1998 study by Daniel Simons and Daniel Levin Watch what happens as the unsuspecting pedestrian provides directions I want you to think about whether you would have this situation happen to you The young man on the left is one of the experimenters. He has approached the white haired man and asked for directions Watch closely as two people carrying a door pass between them And the first experimenter Is replaced by someone else And he didn't notice you think you would have noticed I saw that I saw that like many of the people in this study the pedestrian was entirely unaware that he was talking to a different person Now they go on to show that they even do it They went they got brave and they started changing it from a male to a female and the guys still didn't even realize it People are so focused on helping you figure out what's on your map telling you directions. Don't even catch it going from male to female Anyone use ebay through the years probably not as much as we used to ebay Sorry familiar, you know what color it was You remember when it was this color yellow everywhere So now what ebay did Was one day ebay said well everybody's using white now google's great. It's all white There's a there's other sites that are using all white. We want to go all white as well We want to have an all white ebay screen the background the ui was going to be white They went overnight And changed it to all white and people lost their mind because of that They could not take it that they had changed the color I mean people were mad saying i'm never shopping on ebay again You know they were going on tv shows. It was crazy. People were very irritated and upset that ebay had changed its colors from yellow to white. So ebay said okay. Okay. Okay. Sorry. Sorry sorry Yellow again and went back to yellow And then every week for a year They lightened it just a little bit just a little bit from yellow to less yellow to less yellow to too white And then nobody complained because the people didn't catch the slight changes along the way and eventually It was fine. No more problems crazy, right I when I when I heard this story I I didn't even know that ebay had gone to white and I had to go check it myself to see it I thought I would have you were to ask me what color and I would have bet thousands on it That it was still yellow and I just did I wouldn't have known it I wouldn't want to those upset either. I can take any color on the web page as long as I can buy something Now i'm going to be able to tell which card you pick you pick the one you won't any one of the five Keep your eye on it. I'm going to make your card disappear. Okay Did it Yes All of them changed. Yeah We kept the color. We just changed the suits A lot of people get fooled by that Kids really get fooled by it. I love doing this kid Anybody a kid here? I was We see faces and everything I've been doing this with shoes for all my life taking pictures I got my mother crazy with my shoes You know if you put your phone up to this right now It will it will fit it will start boxing this as if it's a person it sees that as a person And studies show the kids do not like to see that if you if you show this picture to a baby They'll start crying You have a baby don't do that But I did And it does work the babies do not like to see that it bothers them. They like to see this Now use this in the u.s. You probably notice some of these folks But even with distortion you can tell who people are even though you're not really seeing the picture Like this guy anybody know this guy Quentin how about this guy? Yes There's some others may may not know this guy yep and this guy And then how about this guy? Yep And then this guy So your mind fills the gaps You feel the gaps, you know, you you don't see the real picture that you're able to know what that real picture is by seeing We do that a lot. We need to do that as testers Now who is this? Anything wrong with this picture? Yeah, that's how he looks now After all that's going on Now keep your eye on this dot right here Close attention. Now, what's this? Now don't I mean just kind of notice things out of the sides of your you know out of the corner of your eyes But keep your eye on the act the plus in the middle. You noticing something The faces look like monsters And then when you look if you look really quick the monster goes away weird odd. I don't know they There's a cognitive scientist. I put here Matthew Thompson. He did this And he says that they haven't figured out why it does that but you your your mind's focused on the the plus You'll see all kinds of crazy things and there's nothing wrong with the pictures, but you're you're it's changing So you see two different people with two different face types It's it's it's a phenomenon You know what these are? It's not a trick Yeah Yeah Yes Now there's a there's a Canadian psychologist who talks about pattern recognition And he talks about how that Once we realize what something is we kind of know what it is once you've seen a dog you pretty much know what a dog is That gets us in trouble with testers a lot of the times because once we've seen this certain situation We think well, that's the way it's going to work every time and don't don't don't fall into that Because even though you can tell what something is here pattern recognition is good at times But a lot of times it will get you in trouble as well Did you know your eyes have defects everybody's eyes have defects? And your eyes try to make up for it if you'll start that red dot blinking dot in the middle keep your eye on it eventually One two or all three of those other dots will go away You notice that Your eye sees that i put in my notes because i'm not that good at remembering all this stuff But they're micro saccades When when your eyes are fixated about twice a second they make movements called micro saccades That are too small to be noticed your eyes move at most one third of a degree when you perceive the flow the micro saccades Rate is increased and so what happens is your eyes start making up for this and saying i don't supposed to see that and it starts mapping them out Don't stare too long at your screen when you're testing It is going away. How about this one? Same thing here everything on this whole board will go away. You'll just see a wooden board in a minute if you stare long enough Give me a second to do that Good everything disappeared. We're ready to go home now All clean Okay, none of us are as good as all of us. I want you to look at this. This is a gumball Nothing there's no tricks. I don't have anything packed in there other than gumballs This thing has been all over the world And it's been stopped by every airport Person wanting to know why i've got this many gumballs in my bag The new york tsa stopped me and actually wanted to taste them I don't know why she was that brave, but she really did that Now people with me that witnessed that that happened. I want you to tell me how many gumballs are in this Container and i'll come around And i'll let you look you can guess See kind of kind of take a guess think about how many you you think are Oh, wait, don't tell me. Yeah, I gotta write them down 1500 See they're still doing it. All right I'm gonna write these numbers down. So this way keep your number secret. Nobody say one Anybody from the u.s. Know the price is right. We don't play the prices right here. You can't say one You're good Thank you. No Okay all right Now i'll keep this Right here How many I'll take some guesses don't everybody talk at once, but everybody kind of talk at once How many 600 300 Another thousand, okay Okay Hundred There really are that's a good question now. All right, give me, um Just a few more 623, okay You really mean three you think there's three thousand gumballs in there. He's making that up All right, what others just a few more hang on let me get to an even number. I do my math better when I do this At 13 so I need let's let's do two more 1200 Okay, and one last one 400 All right now if we add these up there's 15 of these Did I have my phone? You ever get serious to do this for you? It doesn't do it well All right So we got 600 We got 300 We got 450 we've got 200 We've got 1,000 We've got 900 screwed it up. All right 600 plus 450 That's 200 plus 1,000 That's 900 That's 250 plus 1,000 somebody doing this in their head. That's gonna be awesome Okay, so 15 recover 97 73 divided by 15 the 651 now Would you believe it if I told you that there are 665 gumballs in that container And that at the average of all of your guesses Give us really close to what that was actually in the container That happens a lot This is the law of average is that a lot of people the same number of people that will go over on their guess will go under on their guess and Which leads me to Wisdoms of the crowd Now it's funny when I put this together This this was a an article in 2014 saying about 2018 crowd sourcing will constitute 20 of all enterprise application testing a lot of Providers out there. I need to keep this updated to see if these guys have dropped out, but I think they're still all there Anybody use crowd sourcing for I mean crowd testing No It's a great way to have people all over the world just hack away um at your at your at your software I've I've been a a tester with uh, I've done some work with applause And you test and and they they they give me this thing. So you guys have Krispy Kreme donut here, right? anybody Krispy Kreme donuts It aren't they great I live 30 minutes away from the place where it started And this is you don't get to look like this by accident. I eat a lot of their donuts through the years And I told the person at the airport, you know, I live right Right near down the road from where Krispy Kreme started. He's like, thank you sir. Would you like a donut? It's like I don't care, but but it is true um The reason I say that is you test sent me a link the other day It said would you like to volunteer To test with one of our clients the client was Krispy Kreme donuts They wanted me to test their app ordering donuts driving with the mobile driving to their location And then And picking up the donuts and then they reimburse you for the donuts What great deal testing and donuts at the same time. I did it it worked And uh, I tested their app and also tested their donuts that week Talk a little bit about inattentional blindness. We have 15 minutes and I'll try to be fast Inattentional blindness, we've already talked about some of this already, but um conscious perception is the abnormality It's the same thing as your you drive home and you get all the way home And you don't realize that you missed a lot of things along the way It's not associated with any vision defects or deficits It's just where the individual fails to recognize an unexpected stimulus that is in plain sight Visual stimuli compete for processing resources and when important information loses out the consequences can be lethal we've got situations where um Example where we have an automobile driver who's driving they turn into their Driveway and all of a sudden they hear a thud and they've hit someone on a bicycle because they're so focused on driving home and turning in They don't see someone coming out of the way We've heard situations where Medical staff nurses doctors Don't pay attention and they miss out and they they look at the label They put the wrong syringe and and there's fatal results from some of that where people don't pay attention to what they're doing In a situation with life threatening I don't know of this story I've sort of read this but but I don't know if this is Don't know when this happened or where it happened But talking about the the people on the plane noticing a bulb flashing on the control panel and focusing so much on that That the plane crashed while they were trying to figure out why the bulb is flashing literally Due to not knowing what's going on. You know not focusing on. Hey, we've got a still flight of plane When we're trying to figure out what why this is flashing How do we cure it? Well, there is no cure We're gonna be like this for the rest of our lives, but we can Avoid distractions minimize your multitasking Remember that looking is not the same as seeing How many people have kids here? You know that right they're looking at you looking right at you. They're not they're not seeing you They're just saying okay. You're asking me daddy to to look at you I'm gonna look at you, but I'm thinking of 10 other things But I'm going to play on my iPad 10 minutes and now they're not listening and seeing they're just looking Eyes open does not mean seeing something Make sure you you don't fall prey to that We're gonna talk I want to show you this by stable image now. Which way is this person spinning? How many people think that this person is spinning this way so they're on their They're on their Left leg and their right leg is spinning around And how yeah, how many see that? Okay, how many people think they're standing on their right leg and their left leg Or maybe they're still standing on their left leg. Sorry. Sorry. Sorry. Yeah, how many you think is she spinning this way versus that way? Yep, it changes You guys are really good at seeing a difference. Most people get stuck on that one like when I look at it right now They're kind of she's turning this way, okay But if you stare at it long enough, you'll see or go the other way So it's all it's consistent, but there's somebody come up with a Thing online where you can do this and you can see they once you see it this way with the color There's no way you can see them not turning that way, right? This this one kind of locks you in the same as that one going the other way Sometimes you think more detail you see more detail you'll see things The right way they should be seen but this one could go either way You may see the anamorphic art. You may have seen it not known it I was like, what is this people standing here? But then when you look at it from the other end just the actual view seeing things from the right angle gives you Yeah Oh, are they okay? So you guys seen this for cricket I just learned cricket this week because it's on tv like 24 hours a day. So I've been watching it for three days That was a nfl football fan in the usa and I learned cricket I went to wikipedia. So now I feel like I know everything about cricket. I'm sure I'm sure I don't But I enjoy watching Choice blindness there was a thing on a thing called brain games on in the us a tv show Where they took two cards and then where they had a stack of cards and they had the people guess Which person they had a guy come up and they said, who do you think is the most attractive lady here? And the the guy said I think this one is so he picked number two So they turned over the cards and then they they slid them back and by sleight of hand they replaced the person in number two with With a different lady's picture. For example, the lady may have been blonde hair No glasses, you know and tall and they replaced it with dark hair glasses short And and said and asked this guy, okay now tell us why you picked this person So they turned the card back over he's seeing somebody who he didn't pick But on the show he's going well I picked her because I like her glasses and I like her dark hair and he We find ways to convince ourselves that what we're doing is the right thing even when we may not have Picked the right thing. Does that make sense choice blindness a lot of times We think that when we're testing and that well, we should have gone that way and we start convincing ourselves That was the right thing to do people do this all the time With with once you think that's well, that's what I did. Maybe you go to lunch and you come back and you're like Well, I think that's what I ran. I think that's the test we ran this I think that's what we did So let's just keep going and you convince yourself you've done it this way you fall into that choice blindness There's a time to dig deep sometimes they It may look good on the outside but on the inside it's better with with another situation example, you know what this is And you tell Tomatoes it doesn't like tomatoes But some bad spots Yes Good job You've seen this haven't you? Will you my last present day? How about this one? Yeah Tea bags What this one? No Razor You're gonna get an A in this class today Very well Okay, I quit you in I can't do anymore Now if we were a smaller group I have a puzzle that I give out to folks and I give everybody a puzzle A piece of a puzzle I put you into four groups and what you get is four different bags Every team one through four gets a corner So it took me forever to put this big giant puzzle together And the puzzle I took the corners so that you end up getting team one gets the top left corner So you get only one corner and then it's all broken You know because now team two gets the bottom right corner. I mean the bottom left corner if that makes sense So all four teams have a corner And it's more fun if you actually participate in this and do this but As I knew I would run out of time here One of the things that I used with that is a lot of folks you have a tester's ego You know and you and you say I'm gonna keep I'm gonna keep doing this even though I can't find four corners because I won't do a puzzle if I can't find the four corners me I can't find four corners and then do all the round the part around it And I'm not going to put that puzzle together. I have a lot of people that just struggle and say Oh, you know what I only got one corner I'm going to figure out what to do and I'll start from the corners and work my way to the middle But a lot of times I said to folks Here's the top left corner Here's the top right corner and I found a really cool puzzle that Each quadrant looked really good by themselves Top bottom left and then bottom right But then the problem is nobody looked at the big picture Now a few of my classes that I've given on this somebody will say hey look I think they've got pieces of my puzzle I think they've got some of the parts that I need And and then you see people working together Don't feel like that you as a tester working in the testing organization in the development organization Have to only work with just your team the more you can work integrated with everybody see the big picture Work with your stakeholders work with the development team the test team with everybody It does help you along the way to be able to understand what what you're doing as an organization I cannot remember everything. I Stop focusing so narrowly. I enjoyed this picture. This is near my hometown. Someone posted Stop no no really you got to stop One of the things I thought that was interesting with UI UX design Is when you look at everything that takes place visual design is at the top here Now I don't want you to read all this if you can read this you got really good eyes But but the focus here is about visual design and making sure that that you have Visual design kind of takes I mean what my purpose of saying this is that you find a lot of really good things I mean you you find that visual design kind of caps everything off when you start looking at the product and what you're doing for the Automation we talk, you know this automation conference I want to mention A lot of times automation can really help you Make sure that you don't miss things in visual testing And and it's really good in assisting things that the human brain gets really Finds boring and missing There's tools I don't work for aptly tools. There are people here in this room that work for aptly tools You can talk to them. I'm not here to promote their their product But but but I have seen if you if you take their examples of Um It will quickly show one screen to the other exactly what's going on anybody use aptly tools here Yeah, they'll be familiar with aptly tools. Anybody familiar with aptly tools after this conference. You should really be familiar with aptly tools I'm skipping through a few things here validating the content Visual bugs don't just happen to the small guys ups head issues where this is what they had on their screen The microsoft had issues where go completely off the screen financial times we had issues where Words were wrapping around And and and tools like aptly tools and visual testing tools can find these things for you when you just don't notice them They're small changes small moves in the in the in the wording on the screen or just simple changes. They'll catch Skipping here and the way they do it is they take screenshots They do baseline images then they map the the differences. I'm giving you a very poor aptly tools For us just what I'm not here to do that talk to gill in the room here You want to end in these guys with the apple tools table? It's a very interesting product um Last thing I want to tell you one of the few last things I want to tell you is neuro plasticity Your brain's potential to reorganize be creating neural pathways to adapt There are people that's been proven if someone has brain damage and they have an accident Their brain is damaged and it's part of their brain. It does motor skills or some type of function in your body You can actually rebuild that from You can actually rebuild that by exercising your brain and using your brain and the reason I brought this up is Once you start firing new synapses and you start learning a different way It takes it takes therapy with folks who have physical brain damage To bring back their skills. I say it's testers a lot of times We got to keep exercising the different things that we might be deficient on in testing and figuring out how that you can Build that skill set as well Primacy and recency you may not remember the first The last thing that I talked about But uh, I mean in the middle the thing I talked about through the middle But I hope you remember it a lot through through this presentation but a lot of times we as As as as humans not even testers you'll remember things that happen at the very start of a program or start of a Of any situation and you'll remember the very end, but it's the middle that you don't That you that you miss out on you kind of forget because it comes mundane I had a lego story if we had another three hours we'd do some legos fit some legos together I would tell you quickly because i'm running out of time It and the lego story was I took I took a group of folks put them in five groups Handed them a package of legos without any instructions and these this lego set would do these three things And I made them try to figure out what it was by putting it together And then I said would anybody like instructions you can send one person from your team to come up And look at look at the product and I put this one together the helicopter and When I did that They all went back and they started trying to build a helicopter and then I said how about instructions Would anybody like the booklet and they said yes, please So I give three of the five a helicopter which they had already seen some one of the person had seen it Somebody got the book I give two of them the plane and I give one of them. I mean one of them the plane and one of them both Now the plane looks kind of similar so they didn't notice anything But the boat people got really frustrated lots of times the boat people leave my Workshop and it's always fun to watch them leave. I'm kidding. I don't mean that I don't want you guys to leave stay all day But it's true the the boat people quickly figure out because they can see the difference. We don't pay attention Make sure you look look long if you only get one glance But you can't build in tests without requirements. That's the one thing we found with that lego process was you can You don't have to have requirements, but it's good to know exactly what you're trying to accomplish But you don't have to have detailed requirements every time You may be able to depend on the leader one of the things that the lego thing taught us was The leader didn't always go back and tell them the right things and they didn't always go back and say a helicopter or an airplane They did that well and even when you have a good look sometimes the requirements change as they saw with getting the different changes there So my last slide is this visual testing is critical. We know that We don't always see when we're looking there's time There is a time to look close and there's a time to step back and see the big picture And no matter what you do pay attention Because it'll always help you as testers and in your organization to do so And I know we've got like literally one minute to tag for questions Oh, yeah If you have questions, please Okay, I put someone to sleep I will remember this forever Thank you. Thank you, Mike You said a long day You had to help me set this up Okay, do we have any questions? Thanks, Mike for such an interesting and engaging workshop So I'm pretty sure everyone had fun Do we have any questions? Awesome. So one question I have is how fast you prepare images for your presentation Without picking from internet How would Ask me again. I'm sorry. No, how fast you make Images for your presentation without picking from internet. Um I will tell you that a lot of my images. There's a really great tool. I'm giving you my secrets now So anybody that presents there's a great tool out there called slide bot dot i o About 150 dollars a year But you can type in what you're going to put on the slides and it'll go get the images and Build your power points and give it to you It's one of the best tools a speaker can use and and it's and it's really been helpful for me to have that This is one of my older slides where I kind of pulled in those images myself But um, but with slide bot dot i o You get a free trial for a month if anybody wants to try it's a really cool And I don't work for them either But their product is really cool and I fell in love with it because And I've used it at work. I've used it in my presentations. It's just really creative tool And it does what you can do you just go google it and you could do the work, but it does the work for you So I like that automated part of it Hopefully that answered your question anymore The exhausted this is your longest session of the week, right? Okay Thank you so much. Thank you