 And good evening to all of you, to this wonderful evening about the evening talk about key factors for success for agile testing by none other than Janet Gregory, thanks to us, thanks to her for being here in Singapore for the talk and we have a workshop tomorrow. She just a bit about her, she is a calorie based consultant specialized in building quality systems, passion is promoting agile quality processes in software development. Co-author of the famous book Agile Testing, which I discovered when I joined my first project in Singapore five years back, I tried to, I was trying to learn agile practices and the first book which came up was Agile Testing by Janet and Lisa. So we hope to learn a lot today from you Janet, always. Thank you. How is everybody tonight? Good. So I have one job for you folks, okay, besides sitting and listening and then asking questions after is so, as you mentioned Agile Testing, but I brought along our second book, More Agile Testing and I want to give it away because I don't want to carry it with me on my next segment, it's heavy. So you have to remind me when I'm finished, just before, you know, the Q and A, right? So you have to remind me to give it away. That's your job, all right. So my plug, but what, if you want to tweet, please do Janet Gregory's CA, CA being for Canada. If you want to send me an email, please send me an email. I like getting emails. They're fun. That's kind of all I'm going to say about me today. What I want to talk about, it's called success factors. But I really want to try to not convince you, but to kind of share some of the things that I've learned over the while and really the mindset shift that Agile is. How many folks are working on Agile teams right now? Excellent. Look at that. How many would consider themselves testers? Excellent. How many programmers in the room? Lots. Good, good. How many manager type folks? I see that some people are doing more than one job. This is good. This is good. OK. When I think of agility, right, versus big A or little A, I think it's more not so much a methodology, it's a way of thinking, right? It sounds so easy. How many people have found it really easy to implement? Not so many hands, because it's not easy. And a lot of times it's hard because we really haven't made that shift in our head about what it means. I like Elizabeth Hendrickson's acid test, she calls it. Elizabeth Hendrickson is the author of Explore It. Really good book about exploratory testing. I get the question quite often, how do you know we're doing Agile right? And that's a harder question to answer, but I think this goes a long way. Are you delivering software? Are you delivering, I'd like to say quality software, but frequently? Frequently will depend on your organization, your product. Sustainable pace, and I think that's a keyword that a lot of people forget. So sustainable pace doesn't mean that the programmers get to work there, 40 hour work week all the way along, and then the testers, we delivered everything at the end of the iteration, so testers have to work extra. I was with a team a couple of weeks ago, and that's what was happening. The testers ended up spending the weekend to test things so that they could declare it done. That is not sustainable pace, right? And then of course, the adapting, and that's kind of fundamental. When I talk about Agile testing, I want people to realize that part of that mindset shift is testing is an activity that happens throughout. It happens in tandem at the same time as everything else. It's not a phase at the end, and that's part of that mindset shift. Realizing that testing is an activity, and that testing can be more and much more than testing code. Because when we start testing early, that's what we're doing is we're testing ideas, we're testing people's assumptions. That's also testing, and that's what testing early is about. In our first book, Agile Testing, our final chapter was seven key success factors. And so I'm not gonna talk too much about them, except kind of go through them. The number one success factor for teams, and this is something I think is really important, is the idea that the whole team, and when I say whole team, I'm meaning the delivery team, right? If you have a feature you're gonna deliver, who contributes to that? It's the programmers, it's the testers, it's your product owners, business analysts, if that's what's on your team, that whole team is responsible for delivering. That means that that whole team is also responsible for building in quality. And it starts from the very first time somebody has an idea about a feature. So I use the word feature to mean some business capability, right? Your business people say, we want this, that's a feature. Features have many stories, stories have many tasks, right? So we wanna be thinking about it, but it starts when that first idea comes up and we start talking about it. Really important that that team understands what it is and really starts working on it. The idea of a mindset shift, agile testing mindset. Being able to start thinking more than coding, more than my job is to test. It's really starting to think about how can we work together? How can we change our roles? How can we help each other? What can we do to learn more? It's the mindset shift that we talk about. The third one was automate the regression tests. I strongly and firmly believe that agile teams cannot be sustainable without some form of automation, right? There's unit tests, there's testing at the API layer, there's testing workflows through the user interface. There's all kinds of automation, deployments, bills, right? When we think about automation, it's automated tests. We're usually thinking about the tests that the team does, but there's all kinds of other automation. But that regression suite is really important. The, to have a system, sorry, to have a potentially shippable product at the end of an iteration, in theory means we've run all our regression tests. And the only way to do that is to have them automated, right? So to be sustainable, the team has to work on it together. It's not the tester's job, it's not necessarily the programmer's job. Maybe it's not even that automation team over there. It's that delivery team's job to get that happening. The fourth one was feedback. And when we think of a tester's role or a person on your team that is their passion is testing, those people give feedback all the time. That's their information radiators. Raising defects is one form of feedback. And I actually think it should be a very small part of what they do. Asking questions is a big part of it, right? Trying to figure out who thinks what. So you have two people in a room and you're talking. Chances are you don't think the same thing. Somebody will say a word and you'll have completely different ideas what that means. It even gets worse if you mix cultures so that if you have people in Singapore and then people in, so I'm from Canada in Calgary. So if we sat there and said the same thing, there's pretty good chances that we're not really meaning the same thing. We're thinking differently. So that feedback becomes a very important part of it. Core agile values, practices. It's amazing to me when I go into teams and I see a lot of teams. I do a lot of traveling and when I go into teams, a lot of times they don't really understand the basics. So how many people here have read the Agile Manifesto? Yeah, most people, not everybody. Now, how many have actually read the principles behind them? The second page, fewer, go read them. That's what drives Agile, right? That's really what we have to understand. There's a lot of practices like continuous integration, absolutely necessary. But that's not what makes Agile, right? It's those principles behind it. We have to understand the practices and how they fit together. But the values are really, really important. The sixth one is collaborating with the customers. Not to say collaborating between testers and programmers isn't important. But having that business presence on our team really important. How do we know we're building the right thing? It's only with that conversation with the business being there. And the last one was looking at the big picture. One of the things that happens often on teams is we get stuck looking at the small stories, right? Looking at all those small stories, we forget that those stories are part of a feature. That the feature is part of a system. So having that big picture in your mind all the time really makes a big difference. So these were the seven from our original book. And it was interesting because when we wrote it, we had reviewers, really good reviewers as we were going through. So we send out a list of 20 different practices or 20 different things that we thought were really important. We being Lisa, Mike, co-author and I. We sent it out and we said, pick your top 10 and prioritize them. So most people, and there were about 20 different people that participated, about they all come back with approximately the same top 10. But in different order of priorities, right? Some people thought collaborating with your customers was more important than the foundation of agile practices, for example. But every single person that replied had the same number one success factor that that whole team really needs to buy into quality. Now, when we started doing the second book, we said, okay, we have to have a last chapter. We don't want to do the same thing. But what we decided to do was come up with these, we call them confidence building practices. And these ones really do support a lot of those ones from the first book. So I'll talk about using real examples, the idea of exploratory testing, testing your features, right? Part of that, don't forget your big picture. Learning, because it's very important to continue to learn the idea of sensitivity to context. And the last one is keeping it real. So these are the six I'm going to talk about tonight. Using real examples. We keep talking about using examples. I have a sticker on my laptop that says, use examples. But a lot of times we forget. So a story comes out. We talk about the story and we go, okay, yeah, these are the rules behind them. And we all go away and we understand. Or we think we understand. But when we walk away to go start coding or to start thinking about the tests, there's a discrepancy we didn't really understand. By using real examples, not use a valid username and password. Say, no, this is what a valid username and password. Janet Gregory, password one, two, whatever that is. Real examples, because it's important. That's how people can, you can't misunderstand if you write and say, this is what it means. Real examples. By asking your customer, your product owner, whoever it is that's representing the business, when you say, can you give me an example? All of a sudden, they're brought into the process. They're brought into what you are trying to do. And they will help. Because they have a much better idea that you actually understand what they want. And that's important. Because that's how you earn their trust. Now we're going to do a little exercise. Everybody likes, right? So I want you to find a pair. So one other person, one of you is going to look at the screen and the other person is going to be looking away. Now, this might not work. Hmm. We have a problem. Do you know what? I've never faced this one before. We have screens on the side. Anybody help me with this? Got any ideas? Can we turn those screens off? Thank you. Excellent. Ask for help. People will help. So now it will work. So find a pair. One person face the back of the room and one person is going to be looking at what I show on the screen. It's going to be a picture. And I'm going to give you a couple of minutes, to describe what you see to the person who's facing away. And then after two minutes, I'll bring you back and we'll talk about what you saw. Okay? So find your pair. One person face the other way. Okay, does everybody... Does everybody have a pair? Does everybody have a pair? Does everybody have a pair? If you're left out, then find three and two people, two people listen and one person talks. Okay, everybody ready? Are you faced away? Okay. All right. Go. Very excited people. This is great. All right. So what I want you to do now is the people look at the picture. Was that what you expected? It wasn't what you expected at all. A few. All right. So have a seat for a minute. All right. So those who said it wasn't what you expected, raise your hands again. So can you tell me why it was different? There was a perspective description of a background and a foreground. So the perspective? And that gave me kind of a built conclusion that the houses are kind of much further away. Okay. And then there was a description that the houses look like what kids usually would draw it. So in my context and culture, the rules would be just like that. Right. Hard to describe. How many could describe those rules accurately? Right. Whereas in my culture, all I would have had to say was it's a barmed roof. And everybody would have understood exactly what I meant. Right. What else did you have different in your mind? Who else? Somebody else? Everybody just a perspective in the roof lines. The number of windows on each of them. The number of windows. So did your partner describe the windows? Just describe the main one which is the four windows and neglected the rest. Oh yeah. So what was important to him was not necessarily transmitted. This is one of the things that when your product owners are describing what they want, it's really hard to do that. Now this is a picture that you can all see. When you're talking about something abstract, just think about how much harder that is. So by using examples, drawing pictures. Now if you had a pencil and paper and could have kind of done a mock-up, much easier. Right. So using pictures, using examples, anything that you can do to help your product owner get the message across, be able to get that shared understanding is really important. Because we have differences in what we're saying and what we're hearing. So think about that. Focus on the business and use real examples. This is my sticker I have on my laptop. Acceptance test-driven development. A specification by example. Behavioral-driven developments, BDD. They're all the same premise using different words to describe it. But the premise is always guiding development with examples. So let's create our examples from our feature has many stories. From those stories, when we start the discussion, when we start talking about examples, real examples, then we can start having a better understanding. So a lot of times we have a story. I'm hoping that everybody has acceptance tests. Some people might call them criteria. But the most important thing is really the conversation. Being able to ask the questions, getting examples, guiding development with them. And once we get that, then we have this kind of magic that happens, which is coding, testing and automating. And at the end of that, we probably have some exploratory testing. And then we accept. But it is guiding development. Starting with those examples, starting with that discussion, to get everybody to have that shared common understanding. That is preventing defects. And that's really what we want to do. Because every programmer I know, programmers in the room, do you like fixing defects? No. Spend a little bit extra time understanding the story. Receiving ask. Programmers ask for tests before you start coding. Preventative measures. The second confidence building practice is exploratory testing. Now this is a skill that's not necessarily easy. How many people here think they do really good exploratory testing? Come on, there's going to be a few, no? At least one brave person. Excellent. But it is a skill to be learned like anything else. It's not ad hoc testing. It's having a mission, a charter. What are we going to do? There's lots of different ways to do them. I can explore as a persona. Right? This is PayPal, right? Who uses PayPal? Give me a user. You guys, I'm going to guess some of you, will be a regular user. I'm a young female. I'm just making one up now. Who likes to do a lot of transactions buying new gadgets or something. What would that person test like? Putting on a new hat, trying to say hey. Or maybe it's a hacker trying to make steal all the money that you've got in your PayPal account. Different personas, different users. You will test differently if you put a charter around that persona. A lot of times we will explore around the risks. Maybe security risks or performance risks. Tours, sometimes James Whitaker has a book out on tours. But it's the idea that I'm a tourist. If I'm coming to Singapore, what are three things I must see in Singapore as a tourist? Sentosa. And third one? Chili crabs. I like that. I'm going to tell my person chili crabs, please. Okay, that's a great one by the way. But what are the three things in your application that must work? So you pretend you're a tourist. You design these tours as a tourist and you go, all right, maybe I'm somebody who's coming in and I'm only in town for one day. So I just have to make sure it all works really quickly because I want to get to those three things. How am I going to do that? So you're a travel agent and you're looking for ways to do it. And so you're designing your exploratory test charter. You're looking for completely different things as a tour guide than you would be as the person who wanted just to get through them. So you do that for your application. Most people will do exploratory testing around workflows or journeys. That's kind of the most that I see. But there's lots of different ways to do it. We can take and look at our application and do a great job on getting that shared understanding, preventing all the defects, making sure that we did what it was supposed to do. But don't forget to do that exploratory testing because there is things in your application that you forgot. You didn't think of. So exploratory testing allows you to find those things. It's really important and it has to be built into your time. Or you don't do it, right? Number three, confidence building practice. This is about that not forgetting the big picture. How many like to do jigsaw puzzles? Does anybody here do jigsaw puzzles? A few. Yeah, I like to do them. Unfortunately, I don't have enough time most of the time. This is a jigsaw puzzle that I started. I was going out to our cabinet a couple of years ago actually. Going out for a week and I thought, this gives me a chance. I'm not doing anything, no work. So I took the jigsaw puzzle out, put it on the table. I start by turning all the pieces over so I can see them, separating out the edge pieces, putting them all into different shapes and colors. And then I put the edge together. And then when I was doing this puzzle, there was a lot of pieces that were very similar colors. So I had to take the big pile of one color and divide it into smaller chunks. And I ended up dividing some of those into smaller chunks. And then I looked at the box and I went, what do I want to do first? Because you always take the, you know, I tend to take the most dramatic things. So the first thing I did was the mother's face. And then I did the little girl's face. And then I did the little boy's face. And I finished off the arms because that was the last piece of that color. And then I did some of the background because I could tell where the picture frame was and come up. This is as much as I got done on that puzzle. I didn't finish it in that week. Other things came up. When I was looking at that and I was trying to decide, do I kind of put it away so that when I come back I can finish it? Or do I just put it back in the box and say, next time? And it really started me thinking about what do we do in our development? You know that whole prioritization? The most important things? Well, that's what I ended up doing. I didn't realize it until I started thinking about it. I actually wrote a blog post on this. But I did the most important things. Each piece of this little boy's head is a feature almost or a story maybe, right? But depending on how you're looking at it. But because it was prioritized, I actually put this away back in the box, crumbled it all up, and I was happy because I did the most important things. So we really need to think about what it is so that when we complete it, we get that yes, check, and it's done. What is not as important? So when I do a jigsaw puzzle of a landscape, the part that I leave for last is the sky. And if I never do the sky, I'm okay with that because it's just plug-in trying different pieces, especially when it's all blue. But understand what the business needs and really understand that big feature. We need to think of the system, right? We get stuck at the story level. But how does it relate to the feature? How does it relate to the system? I'm a big fan of teams working together at a feature level, right? Taking that business capability, breaking it up into stories so that the team understands where it comes from. So the team understands the bigger picture, the feature. Why do we want this story? Why do we want this feature? Because that makes a really big difference in how we develop it. Number four, learn continually. I use a lot of pictures from my grandchildren. This is my granddaughter a few years ago. She's much bigger now. But they are constantly learning and not afraid to learn, right? So a lot of times we don't, as adults, don't play anymore, right? Get down on the floor and play with your children but also with each other because we will learn that way. Doing the little game that we just did with the picture, right? It's a little bit of fun. That's how we learn through observation. Watching other people, watching other teams. Somebody actually asked, I was in New Zealand last week and one of the people in my course that was there wants to bring his team. He'd like to bring them to Singapore. He actually asked that. And he said, are there any teams that you know of in Singapore that we would like to go see and see how they're implementing agile? Because we'd like to watch and observe and talk to them. Can you observe other teams and see how they're doing it? But from a personal perspective, learning, there's weekend testers. Anybody ever participate in weekend testers? Nobody? One person? Yeah, right? By learning from others because you're pairing and you're learning different things. That's what we want to start doing. That's how you start innovating. This is, again, this list here is from Elizabeth Hendrick since the slide show she did. But she talked about testers as needing these kinds of characteristics. Curiosity is probably my number one characteristic I talk about that I think people who are testing should have. If you're curious, you will learn anything. Right? So you want to develop yourself. A lot of people are talking about T-shaped skills. What is my strength? From a testing perspective, my strength might be exploratory testing. But maybe it's load testing. Maybe it's something else. It's security testing. But you need to develop that breadth across the top. Knowing about the domain, maybe knowing a little bit about some other types of testing. From a programmer's standpoint, their depth will be coding. I would hope. But it might be different types. It might be back end, it might be front end, it might be dot net, it might be Java. I would certainly hope has a little bit of testing in it. From a testing perspective, I would like to see their breadth have a little bit of technical awareness so that they can talk to each other. It's really important. Constantly learning. Constantly updating skills. Right? Go back to that mindset. If we stop thinking, if testers stop thinking about what their job is to find defects or to make sure everybody does that requirements that they're supposed to, if our job is to help, right? If everybody on that team thinks that their job is to help deliver software successfully, what does that mean? It's a different mindset shift. And we will do anything we can to learn. What does our team need to learn? The fifth one. Understanding your context. Because that makes a really big difference. If I go into one team, and I've talked to a couple of different organizations here in Singapore while I've been here, two totally different contexts. If I went into one team and said, hey guys, you should do what they're doing because this is much better. You should do it. It probably wouldn't work because their context is different. Now, some of the contexts I see is I see more and more mobile and embedded systems, right? Everybody has a phone. I was in a taxi today going to meet a friend and the taxi driver couldn't get us to where we wanted to go because he didn't know. So I pulled out my phone, pulled up Google Maps and said, here, can you show us? Does anybody have... Does anybody go anywhere anymore without their mobile phone? No. So I don't know why the taxi driver didn't have one too, but he didn't. At least not a smart phone, right? But it's in everything we do. And so that testing in a context in mobile phones is a lot different than testing on a website. We're thinking of different things. The first time I worked on an embedded system, I really understood memory to a totally different way of thinking, right? When you're on a computer and you're doing it for your PC, way different than looking at memory on your embedded device. Context is different. You're going to have to learn different things. Distributed teams. All of a sudden, we have communication issues. We have time zone issues. So the kinds of things we're going to have to learn to deal with distributed teams are in a completely different way because we're not small, co-located teams. We have to learn how to... How do I take my tests and give them to the programmer who's... Oh, they're not in Singapore, they're in Calgary. How can we do that? Thinking about the tools and thinking about how we want to make that happen. Different contexts. You have to adapt. Large organizations. How big is PayPal? How many people? I know it's large. Do you know how many people in Singapore? 225 in Singapore. So that in itself is not that large, but I'm guessing they have many 30,000. 13,000. Pretty big organization, right? If your organization is only 200 people, you will do things much different than if you're 13,000 or 50,000 or larger, right? You have to think about that context is different. Anybody work in data warehousing? Yeah. It's different. It's much different. However, if we go back to those principles, those guiding principles, what can we do to make that happen earlier, to test earlier, right? And it's amazing what you can do if you start thinking about asking yourself that question it won't work in my context because we're different. Instead saying, how can I take those ideas and make it work in my context? A different mindset because there's a lot of people doing data warehouse in an agile way now. Very successfully. A lot of times we think about distributed teams in different different cultures. And I know that's one of the things I like about Singapore is that we have a lot of different cultures here. You have a lot of different ones working together. It's quite amazing. But it's not always a cultural country, geographical difference. A lot of times we take and we have oh look, we just bought you. So we're going to absorb you into PayPal. And you have to be our culture. And there's a cultural clash between organizations. Right? So there's lots of different kinds of cultural clashes. Different contexts working together trying to figure it out. Did I miss a number? Number five. What's number six? Maybe it was only, yeah. I'm going to go back to my other screen. Interesting. Number six. And I misnumbered. One of the things that I really, as a presenter, if you're a new presenter, never, never, never do your first presentation to testers. Right? They catch every mistake you make. But it doesn't matter whether it's number six or number seven keeping it real. And I think that a lot of times we forget to keep it real. And what we're meaning by that is just being able to say stop, sometimes saying no. But too many times we get in our iteration and the product owner says hey, I really need this extra story. Can you put it in? What should you say? No, we can't. Or a better way might say we can fit it in our iteration which other one do you want out? Right? Keeping it real. Yeah, hey, don't forget we also have to do exploratory testing. We also have to automate. We also have to do this. Let's put tasks up for those. Make it obvious. Make it visible. The more visible you make things the easier it is to explain. The easier it is to say no and this is why. So you want to make it obvious, right? You're testing results. Don't hide them. Make it visible. That's probably the biggest advantage of Agile is that it's transparent because we don't have to defend anymore. We just say hey, this is what it is. I'll show you everything. Making it and bringing it back to the team if you're having a testing problem. We don't have any test environments that are working. How are we going to fix that? We can't go forward unless, right? Make it team problems. Make it a team issue. Make it real. And really think about what that is. Right? There is no magic. Actually there is. I think there is. If I ask you those are on Agile teams and most of you are if I say who has felt the magic is there anybody who's felt the magic of being on a good Agile team? A few, right? It feels like magic. It does. But it's not really discipline because we don't live in a fantasy world. So when we say things, yeah, we can make that happen. You're giving false information to your product owners. You're giving false information to other people because it takes a lot of discipline to make that feel like magic. And it's when everybody works together. It really is. So yes, my bug was in the number seven. It should only be six. That's okay. I can live with that one. I'll fix it. But these are the confidence-building practices that kind of support those other success factors. And so when we're thinking about it, we come back and we start thinking about that testing is an activity that happens throughout. That everybody on that team is responsible for quality. From the first time we ask the first question, we're starting to test. I think that's kind of really where we want to be is everybody's in that place. Now, I'm not taking credit for this. There's two ladies down in South Africa that put this together. Their company name is growing Agile. But I thought it was kind of interesting that testing manifesto, testing throughout, over testing at the end, right? Yes, we might have to do some testing at the end, but we really want to bring it throughout. Preventing bugs over finding bugs. Because if we prevent them, then the programmers don't have to fix them. So much better. Testing understanding. So that's that testing early. Do we have the same understanding over checking the functionality? That's the idea of going through scripts and making manual regressions we test. Very boring. Building the best session, sorry, the best system over breaking the system. So instead of thinking that my job is to break the system, let's work to build it better in the first place. And then the last one is team responsibility for quality over tester responsibility for quality. Because we cannot test quality in ever. It has to be built in from the very beginning. So I really, really like this little testing manifesto. And so I think that's kind of a really good way to end this talk. And it is about quality and striving for it throughout the whole process. All right, so as 20 after 8, we have some time for questions. Right? And we have a microphone. Oh, you're going to run around with the microphone? Excellent. I think it might be on from before. All right. So I will take questions. You guys, hopefully somebody has a question. At the back. Yes. She's going to bring the microphone or just a minute. Hang on. Oh, technical technical glitch. Throw this to the person. I can't throw that far. I can't throw that far. I'll just point. There you go. You can manage the idea actually managing the agile team. What kind of knowledge is required by the management to really understand the agile tension that is going on? Because I'm here to agile and I'm agile here to manage. So I was going to ask how much of the understanding is required to manage the quality? I think, now what's the most politically best way to save this? I think managers need to trust their team and not get in their way. They have to learn to think about different ways to understand what they're doing versus being in. They have to provide support to their team. So if they have a good understanding, then they can provide support and say, you know, chapter five might be a really good chapter for you to read. You are the first person to ask a question so you get the book. Easiest way to give away a book. But really, it's about having enough understanding so that you know where to point them to. Do you have to have an in-depth? It'd be great, but then that means you have to have an in-depth of everything. But understanding where to point them to is a really big part of it and trusting is huge. All right, next question. Sorry, we need the microphone. Oh, she's got it. Hi, my name is Abhishek. So in the end, we're going to speak of Pyramid. Ah, yes. So is it in the ratio that you maintained is it the three areas? Right. The ratio that you maintained? So the test pyramid, I didn't show it on here. I've shown it so many times. Who doesn't know what the automation pyramid is if I say it? A few people. Can I have one of these whiteboards pulled over and draw? There's a flip chart. Awesome. I can't explain things without drawing. So this is the pyramid we're talking about. This is to think about your regression suites, your automation. So this bottom layer we say are the unit tests, the programmer tests because they're very fast feedback. Face of our regression suite. The programmers maintain that when they're coding. So it's the cost of maintaining is a lot less. This middle layer, API or service layer underneath the user interface, they're pretty fast too because they're not going through the user interface. If you don't touch the database, they're even faster. This is where we want to test our business rules if we want the business to see our tests. This is where our documentation can be. This top layer is through the user interface usually workflow kinds of tests. These are the most expensive because they're slow. They take a long time to run because they're coming through the user interface all the way through the application touching the database and back up. So they're slow. They tend to be the most brittle. So we say push the test slower. We actually have a chapter in that second book that we gave that we talk about the pyramid and some of the different adaptations that people have done with it. But the idea is to try to get the best the fastest tests as your base. That's what you're wanting to do. Push the test lower. Now the question was is there a proportion of tests that you should be maintaining? Not that I know of. I've seen some people try to say that you should have it but it will end so much on your application. But we want to have these tests are about building it right. Internal code quality. This is about making it the middle layer is about making it visible to the business being able to test fast the business rules. And this is kind of a just to make sure that the workflow actually does work. So we want to have the least through here but just because these are really small you will have the biggest number here right? That's why the pyramid kind of works but I don't know of any purpose or any specific answer. Other questions? There's one at the back. I'm going to make you guys run. Can you talk into the microphone? The slides that you show are mainly in technology. Is there any testing tools that the agile testers should have done? Tools that the agile testers should have done. Simplicity. Right? I don't know of any one tool but I encourage teams to find the simplest way that they can do it. Things like mind maps are a really good tool to brainstorm new ideas. I know a lot of teams that actually will use mind maps to store their exploratory test charters and things. So you can do things like that but make it as simple as possible. There's not, I do not recommend tools of any kind specifically because it will very much change on what you're trying to do. So we are talking about testers in Nigeria. There's no specific role called a tester in Nigeria. So yeah, some people said that. Yeah. Even the practices are the same. We have been practicing for quite some years on that. We don't have any specific role called a tester. We have a kind of cross-functional team from time to time. Without no work roles you can jump into the testing area from this perspective. Right. I hope everybody does testing. Yeah. So how does this work? Yeah. I'm not one of those people that say everybody can do everything. I think that's a fallacy. Because I think if we generalize, if we make it too general nobody can do anything really well. So I like the term generalized specialist. So I don't care if I call me a tester, though I'm proud to call myself a tester. I can be a developer because I think the teams are all developers made up of lots of different specialties. So whatever you label yourself I am going to guess that you have people on your team that are passionate about one thing. Right. If a person is passionate about something that's what their core competency is going to be. I started off as a programmer. I went to university. I learned to program. I programmed for my first six or seven years. I knew that I would not do that for the rest of my life. I did not love it. It was not my passion. I was okay at it but it wasn't my passion. When I moved into testing that opportunity I found it. I read testing blogs. I read books. I read this. I don't remember the last time I picked up a C++ book or a Java. It's not there. I will keep up my skills to the point where if I sit and talk with a programmer I know what he's talking about. I can read his code but I would not do it again. What is your passion? Don't take that away from people. Don't do that because they will become a generalist and they will stop reading anything in my opinion. That's my opinion. Be very careful. I'm not one of those people that think everybody should do everything. We have passions. Not everybody agrees with me. And that's okay. Hi, I'm Victoria here. Hi Victoria. I'm very interested in the 7th test event development for the flow of the event. I just want to understand how it is to be applied in a job. Because when we start usually for each screen we will talk about the user stories. The manager will do the evaluation of the user stories. When is it the place for sharing and who is supposed to in which phase because in pure agile I don't see that. And also there is this acceptance test that you draw a beautiful diagram which I love it because this is exactly what we need. In pure agile I don't see that being shared. There is no such thing as pure agile. It isn't. There might be pure scrum. Yeah. And you're right. But scrum does not talk about practices. They don't. They talk about bringing how you get stories from the business into the rest. And they talk about how you work with it. But it doesn't talk about programming practices. It doesn't talk about testing practices. So you do have to go out. And so there's lots of books that describe acceptance test driven development or specification by examples. Goiko's books are really good. So there's ones out there that describe. I think it starts and I think that it's really important that the product owner doesn't go off and make up the stories all by themselves. I think that the team needs to be involved because that's how you get flexible stories. Is the team is getting involved in doing that. So as you're doing your backlog refinement sessions that's when those conversations happen. Right. So the proposal would have the requirements together with the rest of the team. I firmly believe that. How about the high level acceptance test would that be inside your user's story as part of what you present in your user's story. So yeah, you have your user's story. It's one sentence. Right. One sentence. Not enough. So when you're talking about the user's story and your backlog refinement meetings my preference is to call them story readiness because that's what you're doing is getting the stories ready. When you're talking about that story, you have the conversation. I like to start with the acceptance tests. What is the scope of this story? Can you give me an example of desired behavior? What do you expect? That becomes that acceptance test the high level acceptance tests and have the conversation exploring examples around that examples of the desired behavior examples of misbehaviors what if kinds of things and it comes out of those conversations but you want your product, the product owner owns the acceptance tests. Doesn't mean they have to do it all by themselves. Right. And that whole conversation you should come on my course. So this is more like the conversation rather than everything you prepared inside the user's mind. Yeah. So the explore examples of these acceptance Sorry. I'm going to stop because we could have this conversation all night and I think there's some other questions but I can if you send me an email janet.ajaltester.ca I will give you lots of things to read lots of things to read. Thank you for your interest. Yes. So the dating I'm still reading your first rule the dating to the 4th order Pardon? The 4th order for Q1 is Yes. I believe that Q1 is more for programmer kind of testing. Quadrant one is for programmers yes. So actually it does the tester who TDD together with the programmer. Pair with the programmers on doing unit tests and things. I've also seen testers not necessarily pair with them but help review to say hey sorry if you're the person I would be glad to sit with you and pair and review your unit tests. Did you think of this? What about this? So I've seen that as well I do believe that the programmers should be responsible for their unit tests because that's their safety net for their own code so I personally don't like to see even though I know teams do this say can you go write my unit test and then I'll write the code I don't like to put that off I don't want a tester to write those unit tests I would rather the programmer do but if a tester can pair with them that's great. I think that's absolutely great. I don't see it very often though. This is for the important pair programming that's why you mentioned test so I enjoyed that kind of collaboration and interaction because you mentioned that the tester not the tester, the programmer utility is more thinking of the design. So if a tester pair with a programmer and he can write a test or he can't move an implementation so what exactly is the role when it's paired? So it's interesting I have never seen particularly that a tester and programmer pair on that right? I've heard Microsoft used to I don't know if they still do but used to have a nest at test, write the unit test give it to the programmer and let the programmer pass it I don't like that at all but I've never really seen them pair on the actual development right? Mostly because if you have a tester on a team they tend to be, I would rather see them writing the business level tests and having them prepared for the programmer so yeah but I'd like to see it happen and you have an experience report send me an email and tell me about it because I think that would be really interesting so basically what we have done is more like it's incremental so it's just maybe you do just what the test is because you have a character it's just inside it so one is an aggregate, the other one is right then the person who write the code immediately convert to a tester and then the person okay is it working well? working together cool I like new experiments I think it's great yes so everybody that put up their hand that said they felt the magic they knew what I meant every once in a while and I don't have to explain the people that went I think maybe I feel it maybe they're the ones that have been on the edge and almost felt it but it's not there the ones who go what do you mean you're not anywhere close to doing to being agile right because you can't imagine what it feels like I can't explain it it's the cadence that a team gets that everybody is working together you get good retrospectives you're working together it's a feel good kind of team and so if you've never felt it it's really hard to explain but if you felt it you know what that means I've been very privileged to be on several teams that I felt the magic in a lot of people get one or two in their lives and I've been on several so it's it's a nice feeling sorry but it does exist that feeling so work towards it alright yes just on your last couple of slides I've got about making things a a team problem and shared responsibility team responsibility for testing do you have suggestions on how to achieve that especially with people where traditionally testers are responsible for all these things it sometimes takes a long time to get that sometimes it takes a lot of courage so one example that I use is we were implementing our top layer of our pyramid so we were writing some tests through the workflow and we were using a tool called ruby and water and when we were doing it we were spending a lot of time on maintenance the first iteration we just we put in a couple of tests and we're like we're spending too much time and so we took it to our retrospective and we said hey team everybody we're spending a lot of time on these these tests because you programmers are using dynamic IDs and we're having to change these tests every morning when you add something new what can we do about it that's the key what can we do about it so keep asking that kind of question so the programmers responded in a very nice way because they wanted to help and they said well we can actually start putting names on those objects will that help and we said yes so they did that and it did it helped a lot it cut down our maintenance but we were still having the backlog next iteration the retrospective we said so much better but it's still making us do work is there anything else and as somebody and this is where that technical awareness comes in so the programmers of course say that's a lot of work right programmers it's a lot of work I don't want to do it and so my answer to that or my question was don't you guys have a search and replace function in your IDE and they kind of went right and I just said how long would it take so we did the math it took them a half a day to make all of the changes and retrofit all of the object IDs versus us spending that 20 minutes half an hour every morning fixing our tests it was a no brainer because we sat there and we worked but it takes a long time for the team to realize that you can work together to make those things to make that happen but it's a constant how are we going to address it so if a tester says it's my problem I've got to fix it you've taken the monkey don't take the monkey that's part of keeping it real as well right chocolates work really well with programmers too alright one last question what time do we have to be out of here where are we at so they're going to kick us out so as we know that if we can expect last moment requirement change so in the beginning the product managers they come up with new features and they have this abstract picture in their mind and they come up and they inform the developers that this picture is going to look like this but at the end they say let's do it like this it's not looking that great so last moment requirement change we see in our day to day life so we may lose we may lose to understand how critical that that product is because the requirements are getting changed so we may lose interest or may lose the criticality of the products so so changing requirements one question is the product owner changing in the middle of an iteration oh yeah don't do that don't do that right if they want to change something in the middle of an iteration that's keeping it real right let's stop the iteration you want to change it that much stop the iteration let's have a new iteration planning meeting keep it real if they're doing that in the middle then they're disrupting it's a team responsibility to say okay we're going to stop let's replan it keeps them from doing that so yeah that's part of that whole keeping it real alright I'll turn it back over to our organizer Riju I think software delivery from Singapore is going to improve from tomorrow I hope so we've enjoyed it thanks so much thank you