 Welcome to Agile Roots 2010, sponsored by version 1, Rally Software, Virio, Amirisys, Agile Alliance, and Xmission Internet. A fresh approach to certification by Amid Sidki and Aleister Coburn. Okay, certification is a hot topic. Who has been following any of the certification stuff floating around in the industry? Where are you guys following it from? Where are you like seeing stuff about? Well, I didn't look recently. I was something that started about 20 years ago. Okay. Oh, I've been watching it from the Agile Alliance perspective. I've been looking with some kind of, you know, lack of certification kinds of things like that. Okay, we've got one more. Yeah. Okay, how many Agile Alliance people here at your Agile Alliance? Good. So the background on this, there's a couple of different forks, a path to come in here. Back around 2004, several of us, and I would put David's fans, Sanjeev, Augustine, and me, we're talking about, this is pretty PMDI stuff. Course curriculum, plural for Agile. And so the way we were designing it was with sort of learning outcomes and levels and branches and specialty degrees and certificates and stuff like that. The PMDI came along, and then the APLN came along for those, that's Agile Project Leadership Network. So PMDI is the Project Management Declaration of Interpreting. Which is kind of a post script on the Agile Manifesto from the product management, project management, line management perspective. Right? If you take out software and you take out, you know, some project where you get. And we wrote down some stuff and problems in on that. And then a group formed around that called the APLN Agile Project Leadership Network. So we took that idea of course curriculum and stuff forward and built up then had various competing ideas on how to do it. But the one that leads forward into today was the idea that you would have sort of a Boy Scout merit badge kind of a system. So you take courses, you collect merit badges, then you get, you know, first class, then you get second class, then you do an Eagle Scout project. That type of a thing was the one that was proposed about 2005, 2007. And the APLN declined to use it for various reasons that, you know, and Agile Alliance at that time on the way you guys are now. Some of the people are pretty vocal. They didn't like any of the certification stuff. Period. Okay? That, so the DSDM community in England picked it up and they've adopted. So Agile Alliance dropped, so to speak, this had left it there. The DSDM people picked it up and they have it now as part of their certification sequence or whatever. And then Ahmed picked it up and said I want to go forward with this. And if I don't mind telling this about that he works mostly in the Middle East. In the Middle East if you don't have certificates and certifications and ladders and chains and stuff, you basically are a non-player. So he needs this in the Middle East, came back to me, said, this is the kind of thing I'm thinking about. And I said, that's what we've been thinking about since, you know, 2004, 2007, makes sense. He added kind of the Eagle Scout project at the end, which is very nice and kind of a bow on it. Part of the other epithesis of this is frankly people are sick of the certified SCROM master certificate, right? Breathe for two days, get a certificate. I'm a tough task master in my class, you have to breathe for three days. If you stop breathing like halfway through, you don't get the certificate. So you have to finish the whole three days of breathing to get the certificate. Can you sleep? Yes, sleeping is okay. You just breathe, man, just breathe. And I will take some valuable time to make fun of both the PMI and the SCROM alliance. I buy a certified SCROM trainer and I make money from handing out SCROM certificates for this quarter of a year. But the PMI has this fairly expensive, fairly rigorous, lengthy, painful thing you go through to get a project management professional certificate called PMP, which attests to nothing about how you're going to be on a project. And what I like to say is the agile people, the PMI people, because we can give you a workflow certificate in just two days, instead of forcing all that long work. Okay, that had to be said. So the thing is the trouble in a certain sense is that the certified SCROM master certificate has taken a market value. So I don't think SCROM is not a bad thing. I wasn't going to teach it. CSM is not a bad thing. The things you learn in the SCROM classes are not bad things. But it's a small piece of the pie. Okay, so what we're going to be showing you today is a question of, and our desire is to share what the pie is and let people collect their merit badges and get the first class, second class. People scout stuff as they go through that. What is also interesting is I'm starting as I travel internationally and I go to some company, last most recent one was Valentine King, France, and they're frustrated with the CSM certificate meaning anything in their two-day courses, certificates don't be up for anything. They're getting ready to start creating those certificates. So I say to them, why fight over the certificate? Let's set up the learning outcomes. You say which of your courses cover which of the bits of competency or expertise. Let's level the playing field on the certificates and compete on the courses like you used to do. And that saves them a lot of trouble because they don't frankly want to create the certificates and fight against that idea and the question of certificates and stuff. So that's the direction we're going is can we do that. I gave the historical basis of where this came from, the shape of it. It's going to be basically competencies or learning objectives and then this is sort of some kind of a big in-person test at the end to get the Eagle Scout equivalent like that. The intention is to work with the PMI and the IIDA that's the business analysis group and work out what would be inside of these job descriptions as learning outcomes or competencies and then let people say these courses hit those competencies and work that way. That's the general direction that we're going. Does anyone want to take a question before I hand it over to Ahmed? We'll show more details please. So I see the international group consortium for Agile and I guess I don't know much about that but is your certification that you're talking about associated with that? So I see Agile, the national consortium Agile is our intention. Non-profit, international, work with the group, set up competencies, had a clearing house for the courses, registry, let people accumulate their stuff. Those of you guys know me in the way I test people, I ran him through my set of questions and he came out clean and I said cool, I'm on board. So I'm on board, that's it. We haven't created the corporation yet, due diligence, funding, cost operations, pricing models, a lot of that. So this is what you're getting, it's an intention of where we're going and it's also what we've got, what the thinking is, right? We have one back there? Yeah, I have two questions. The first question is are you aware of the Agile Development Still project? I am, not very but I am. And I would say that since I'm not very, you'll see where it would enter into the discussion in here. And my second question is why the international consortium for Agile versus the Agile Alliance? Agile Alliance turned down this issue back in 2007 so we just decided we were going to go for it. That's it. I have personally been working on this for six years and we keep offering it to the different people and they keep saying no, so fine, let's go. Yes, please. How do you get attached to the stigma that the Scrum Certification has? Very good question. And there's two things that are kind of big emotional poppings right now. One is the fact that the Scrum Certification is only two days of reading or three if you've got an agile meeting. And the other is the fact that both the Agile Alliance and the APLN declined this approach three or four years ago, respectively. So the reason I'm standing up here instead of Ahmed is because this is a tough nut. And I think this thing comes out clean and fair. My view is the way you deal with the stigma around the CSM certificate is you have a larger space and you let the market decide on the meaning of the different certificates. So you take CSM, you'll get like two merit badges out of 20. For example, and that properly frames the CSM, the value of CSM is two out of 20. So that's fine, you've got two out of 20, cool. The merit badge concept allows people to spend years rolling their levels because it might take two, three, five, so who cares. And the person goes from job to job, they say here's where I am on the journey. This is what I've accumulated so far, and I'm like that. So my view has long been that binary certifications are actually, there's a negative to them, but it's a downside, which it doesn't allow. And I go back to Cisco, Cisco's got three levels. I don't know much about Cisco, but my brother, my son, my brother-in-law, my brother-in-law, cousin, nephew thing. One of them is nephew. He's a nephew. I guess he's a nephew. Went to the Cisco, and the first one was easy. And the second one was pretty easy. And the third one kicked his butt and he had to take it three times. It was like hard, hard, hard. He's got Cisco level three certification. It means something, right? Eel Scout means something, right? So the way you get away from this, you know, PMPs, it's hard, but it's still binary. So if you have this gradation, then it can attract some meaning. And a little friend of CSM's, all of CSM's could have their two free merit badges, you know, straight away as far as I'm concerned. That kind of a thing. That's why you paint the bigger picture, you make lots of steps. And the idea is to let the market do the proper market kind of work. Course versus course, and level versus level. That's where we're going. Certainly. So I've been on the Avalon board, and I don't recall coming to the Avalon board. I think, as I recall, if you call out names, that the people are not called names, because they don't know enough of them at the time, that we're so vehemently against the concept, if you recall, that we were on the Avalon board. So you let it never even came in, and then you let the topic in the board, which is another way of explaining it. Because if you made the Avalon position on certification as misinterpreted, although I think it's because of the Avalon board, who happened very strongly to any certification, but the official position was that any certification that's out there should be difficult to obtain. That's their opinion. And here's the mind field with the opinion that someone says, like that, a certification should be different. I don't know that I knew about it. You could have something, right? And this is the pattern that I accept. I buy into this what you're about to see. We'll be interested to see either what the objections are, who says, buy, gummy, I'll be in my grave before you come out with this, or who's on the Avalon board. Be very curious to see where we'll go. For us, this is very frightening, right? This is a heavy emotional topic. So I guess the follow-up to that is, if there's enough people in the Agile Alliance to say, oh, we like this, are you open to the Agile Alliance being a partner in this? Yeah, we're open editing the moment because it has not incorporated. Exactly that we're partnering with. We're talking already in detail with the IIVA and PMI. If we don't need to re-emerge the wheel? No one wants to. I'm not sure that that's ideal, right? I just want to know what we're hoping. Yeah, right now it's all open, but I'll just tell you, I, for one, I'm frustrated with trying to give it away and having it turned down, so I'm rolling. Todd also has seen the APLN competencies. Did anyone see that early work from 2007? So you'll see a lot of overlap, because that was the starting base. All right, with that, I'm going to look, I'm going to show that to the students. Thanks. So, like Alcer was saying, it's a step-by-step model. There were two things we were very conscious about while designing the whole certification program. One was looking for skills, the other was transparency. So, to move along, there's basically three steps. This is the overall structure of the certification program. Three steps. First is the fundamental space, then the focus, then the certification space, although for each one of them in detail. So let's start with the fundamental space. So, as Alcer was saying, this is built on learning objectives. So, what IC Agile, the International Consortium for Agile will do, is provide the learning objectives. So, people that want to finish this phase need to know one, two, three, four, five. We actually have 32 of them. I'll show you that in a minute. Then, the instructors will prepare their courses based on these learning objectives. You can prepare whatever course you want based on these learning objectives. And at the same time, each instructor is asked to provide their own assessment method for the 32 learning objectives. Let me pause here for a second. So, while we were looking at this, and again, the whole idea of certification, it's not just about certification, it's how it's done. I mean, hairdressers are certified, the doctors are certified, and there's scuba divers are certified. There's different kinds, but how is it done? So, we looked at many different models, starting from universities, driver license, scuba diving, and we tried to take what makes sense from each of these to what we're trying to do. So, one of the things is having the same person teaching provide the assessment. And that was a point of discussion. Is that, yes, no, is it okay, is it not? And so, we looked at the university model where the professor teaches. So, he gets a set of objectives from the department or whatever. They teach the course, and at the same time they assess the students through whatever means they come up with, either through a project, through a test, and whatever. So, each instructor is asked to provide, how are they going to assess that the students in their class know these 32 learning objectives? Or, especially for the fundamentals phase, we've prepared an exam. So, if you don't want to go through the hassle of preparing that, especially for the fundamentals, here's something, build on it, tear it apart, add to it, subtract from it. But, at the end of the day, you need to show, discuss with IC Agile, how are you going to assess that these people know these 32 learning objectives. So, how we've done it, an announcer was saying, I work a lot in the Middle East, and again, there are cultures that this certification thing is in their blood, and believe it or not, it's more than you think. Just cross the Atlantic and you'll find out. So, we've actually piloted this for the past six months in different countries in the Middle East, and how we've been doing it is quite interesting. And let me just, I guess, highlight that very quickly. So, the exam is handed out at the beginning of the class. So, we start the class, the exam is handed out to everyone, you can imagine the reaction on their face, they have no idea they're coming to learn this, but the experience of going through the exam, they now know what we're looking for. In addition, they already have a printed copy of the learning objectives right beside it. So, they know what they need to get out of the course, and here's how we are going to assess it. So, it's sort of like TVD, and it's nice to start to introduce that concept. At the end of the class, and this eliminates something I think very important, people focusing on the exam during the course. Is this going to be part of the exam? Is it not going to be? You've already seen it. That shouldn't be your concern. And so, they go through the class, and at the end, the exam is solved in groups. And that group result, that group result is what they're assessed on, plus a group assessment. Part of the learning objectives, and I'll just show them real quickly here. These are 32 learning objectives. I don't expect you to read them right now. They're on the website, so you can go take a look at them. But some of them say demonstrate. Some of them say demonstrate. Some of them say explain, and discuss we can assess those through exams. That's not a big deal. But demonstrate how you assess demonstration through exam. So part of assessing these demonstrates is they have to do the exercise in groups, and everyone in the group has to assess each other. So part of that group assessment is assessing their contribution to the group exam. And so these two inputs are taken, and again, this is one way of assessing it. There may be others. So you take the assessment of the students is based on the exam and the group activities. And so there's a little bit of applying it, assessing that you've actually applied it, and the knowledge. And remember, this is all in the fundamental space. What do they get after that? They get, if they pass quality, they get a certificate of agile fundamentals. Took the class, I know what it means. And it also, having these learning objectives published, and they're open, they're transparent, even people that hire them know what is expected of these people. These are not masters of anything. This is what they know. They may know more, but that's up to them. That's not what we are saying they know. And they become an icy agile associate, just a member of the consortium. So we went through these. So this is the fundamentals phase. So this is just step one, basic agile knowledge. And we go through mindset, how to plan, all those kind of things in the learning objectives. And at the end, if you want, I can read them back. Next comes the focus phase. And with the focus phase, basically, we're looking at sort of an open architecture where organizations or disciplines or roles that have learning objectives can just plug right in. So we finished the prerequisites. Now, someone is a project manager. There's agile project management learning objectives. There's agile development learning objectives. We need to know things about design and and and. There's coaching learning objectives. There's product management. There's testing. And so for each one of these focus tracks, a set of learning objectives, just like the one you saw with the fundamentals, are defined. And so, let me zoom in here a little bit. So again, the instructors are asked to prepare courses based on these learning objectives. And here I add the word courses because something like the agile development track, specifically, again, being a developer, I doubt you can finish all the learning objectives in one class. Either it be a five, six day boot camp, that's fine. So again, there's flexibility here. It's either a course or courses. All right. And so I made to find one course that tackles 10 of the learning objectives and another course that tackles, you know, the next task. So I'll have one class on design patterns or simple design and so forth. So instructors prepare their course or courses around these learning objectives. And again, they're asked to create an assessment for these learning objectives. Now, again, it can be anything through a project, through, I go to the extreme, someone says I will sit down and interview every one of my students. Okay. Now, there's a point here. Again, I want to emphasize this whole separation of instructor and assessor. We preach trust in the agile community, right? And when you look at the separation, part of the reason behind it is, can I trust you to actually give the course and at the same time assess objectively? So the criteria for picking instructors is very important, but once they're in, there is an assumed level of trust until breached, right? But so they are asked to prepare their own assessment. The course is conducted. People go through this assessment and then they get a certificate. So I finished simple design, again, not through the consortium, but they get a certificate of completion of that course. What happens here when participants cover all the courses or finish all the learning objectives in a certain focus track, at that point they get a certificate of agile specialization, now they know something more about agile project management and they become an icy agile professional. Okay? The next step, so we finish the fundamentals phase, we finish the focus phase, the next step is the actual certification phase. So along the way they're getting, like Alistair's saying, these merit badges, they finish this, they gain that and so forth, now let's move to the certification phase. So remember, we want a skill-based certification. So part of what we're building on here is the hiring model. When you hire someone in your organization, that's something you're really looking for. You want someone you trust, someone that's competent. So we looked at that kind of model. So you apply for the certification phase through an application. And the application highlights your experience around this particular focus track that you're going through and there may be, and I'm saying maybe because it differs from track to track, there may be certain prerequisites to even apply for the certification phase, meaning the agile project management track may have requirements that you've applied this for one year. The agile testing track may not happen. I'm just giving examples. These haven't been decided yet. But there may be prerequisites. So this application that looks for experience and looks that you've fulfilled these prerequisites, now each applicant is given a phone interview, just like the hiring process. Let's talk to real people here. And with experts on the other side of the phone interview, they will get a sense that this person knows what they're talking about or not. And so the phone interviewers to look through the application, discuss it with them. So the application isn't just submitted and rejected without any kind of conversation. Or you just get feedback. No, there's a discussion. So tell me how you do it. So who did you take the courses with? What was most interesting? It's an interview, right? And this will really bring out a lot of things. If they pass the interview, then the next step is what we call a hands-on immersion. Hands-on immersion, this is the interesting part We've been talking to or the idea is to either create our own conferences or tag along to conferences like the Agile Alliance Conference, SQE Conference, and basically have you guys have been to the Agile Alliance Conference? Raise your hands? Okay. You guys heard of programming with the stars? Okay. So it's sort of like that where let's say we have a developer coming out from the development track, test their coach, and they create a team. Okay. And this team will be on a stage in a room, whatever. But they actually apply, all right, with a real customer there, they take a slice of the problem and they actually develop it. Now, sitting in front of them are assessors. So people that will look just like Dancing with the Stars and so forth, they will look at them and the idea is if an expert is sitting there, it will take probably 10, 15 minutes and they will know if the person in front of them knows what they're doing or not. Right? It's the blink concept. You'll get a sense. But the idea, the idea is to leave them for four hours. The assessors will go around maybe looking at different teams, but for four hours, these people are actually developing. During this conference, the idea also is to have people that want to see an agile team in action. This is right there in front of them. They can go, they can see, they can see how people are applying this in real life and it's a very transparent process. Everyone knows who's up next, who's being tested, who's being observed. And so before we can say someone's certified, if anyone wants the challenge, come and take a look at them. All right? So with the hands-on assessment, they are assessed for the application of their skills during this phase and at the end, if they pass that, they receive a certificate. Yes? The hands-on immersion, if I take it, is intended to be as public and transparent as possible? Yes, sir. Thank you. And that's part of it. So no one will suddenly be a certified agile project manager and no one is afraid of it. So if you want, here they are, they're going to be doing their thing, come take a look at them and so forth. Are you intending that the hands-on, if you have multiple applicants at any given time, would you be wanting individual separate immersions? So the idea is, you would, so like I was saying, this is probably an additional day to the conference and the extension, where you would go and it's all stages of programming with the stars. It's not one. Right. So there's multiple ones because of the number of people that will be applying. And we're also looking at sort of adding this because the idea for it to be international with XP days in different countries, different agile conferences. So we want to provide that, I guess, venue for people to really step up, perform and show what they can do. So you might have say four developers, sort of four people who are looking specifically for the development focus, working together in a single session. Yeah. All four of them. Yeah, definitely. Thank you. And so they get a certificate of agile expertise and become an IC agile expert. Sort of, if you want to look at the overall picture after I just, so you have the fundamentals of agile, you finish that and become an agile associate, you have the focus tracks and as I was saying in the focus tracks and Alistair talked about this, the learning objectives can be, we can develop them as IC agile or in association with PMI, IIDA. Again, we don't want to reinvent the wheel. If someone wants to step up and help, great. And so coaching, we've talked initially to SEI about agile CMI. Then we have the certification phase, the phone interview, the hands-on immersion and then the last step, which is becoming the IC agile expert. Here's another way of looking at all these pictures are on the website. So, and we can kind of, so they take the fundamentals, again, we've been through all this. Just the last designation here in IC agile fellow, so once the person has become an expert, remember, this is, I don't want to call it volunteer based, but there's a lot of contribution from the community with the phone interviews, with all this. It's an ecosystem because at the end of the day, if we don't take care of the whole certification, then we are going to be the ones that are harmed at the end. So, by community contribution, helping out, being part of the assessor panel at the conferences and so forth, and peer nomination to become an IC agile fellow. So, we talked about the IC agile fellow track, and I'm sure a lot of you are thinking, so what about the instructor track? How do people become instructors? And so, the idea is this. Again, it's a min and application. You want to become an instructor. And, while we're doing the instructor track, a couple of thoughts came to mind. We don't want it to be just an easy process of how you're in, and at the same time, remember, there are a lot of people doing amazing, agile stuff out there. They're probably better than me, better than others, that don't have any designations. They're not a CSM, a CS anything. All right? But they know, they know what they're doing. So, I'm going to submit an application, and then the application is reviewed by a set of experts and fellows, and then each applicant will, again, go through the phone interview. So, again, I'm repeating. As you can see, there's a lot of community contribution here. All right? But again, it's not for free. There's honorariums, plans into the ID, and so forth. But the idea is, they will talk to people. People will assess them. People will, does this person, how many years of experience, and that interview will let out a lot of things. So, either, through the phone interview, we say, okay, maybe you need some mentoring, you need to practice a little more. Here's a couple of people, you need to talk to you in your locality, and so forth. If the phone interview goes fine, remember, we have these venues, which are the conferences. So, the applicant is scheduled for a face-to-face interview at the next conference. Whatever that next conference will be. All right? And then, they're required to present a topic of their choice to the people. And so, again, it's by application. You want to be an instructor, get up there. There will be a couple of people from the ICA Agile site to assess your teaching skills, your knowledge, how you can put questions, and at the same time, there's regular attendees that will give feedback. Based on both of these feedbacks, the instructor will be evaluated on both of these feedbacks, and a face-to-face interview they either pass. And when they pass, they become an authorized instructor for the fundamentals of Agile track. And then, there may be separate interviews for if they want to be a development instructor. As you can imagine, that's a whole set of questions and applications and so forth. So, this is the basic idea behind the certification program. Questions? Yes? I have a lot of questions. That's why we're here. That's why we're finished here. And I'm going to address these and both you and Alistair. Come on down. I was enjoying this. So, let's see. I'll ask these one at a time and let other people jump in maybe with me. So first, what you're going to do here if your success goal is to create a power structure. And how do we prevent that power structure from stifling evolution and innovation in the Agile space? Because you're going to establish learning objectives and really say, this is what Agile is and this is what it's not. How is that going to allow Agile to continue to evolve? Nice question. The first answer is just that I'm so allergic to some of those things that, you know, while he was talking about the develop track and the professional developer Agile skills thing and so on. I get kind of nervous when someone says that you must do an XP practice in particular where you are not doing Agile. That kind of thinking scares me. So I'm almost like the first ombudsman on that. At least at the fundamentals track. For instance, I think the most interesting one was had we done this three years ago, there would have been about time boxing iterations in spring since the arrival of Canben. We've had to reword that one to take it on. So those things will, you know, that's where if you have the experts and the fellows there, these are the people who are up on the community. It will catch the drift. Let me add to that. The learning objectives need to evolve. Anything. I mean, if anything becomes stagnant, it stops there. The evolution stops. But the idea is the learning objectives will capture now and then they will continue to evolve as the industry evolves. That's one question and we've got two other hands up. I do want to clarify that question. Yeah, please. Go for it. Go for it. That's a good question. Go for it. So the concern I have, I mean, this is the best about certification that I've ever seen. So I guess what I want to know is do you have you created mechanisms for that evolution and have you thought about the human nature of protecting what you've already got? Those people who are already, you know, can I say for myself that's a great point. I personally haven't thought about it, but I will just take that on. Part of the idea of having, as I say, the experts and the fellows in there being leaders, you know, up on the field that will cause a natural growth. Power structure is inevitable. You know, I mean, just as Todd brought up Agile Alliance was nothing and then now it's a power center. Scrum Alliance was nothing. It's a power center. PMI was nothing. It's a power center. The presence arrival of a power center is scary to people, right? And people who have power and people who aren't in power are scared of more power centers showing up. So I think that's very good. We had not discussed explicit mechanisms for evolution. That's a really great thing. For Ahmed or anybody to say, of course it will evolve, it's kind of acutely, you don't have the mechanism. And I was thinking of the experts and the fellows naturally doing that. But you're right. So that's a great point about introducing a mechanism. We've got two more questions in here. Well, just to be fair, I'm content to hear every question that you have. No, I do. I have a couple of questions. The James has questions. I'm definitely interested. I just want to ask you. Yeah, but James could keep us here past wine and beer time. I think it's not a certain life lesson. That might be the most beneficial approach to this. So we have, except you missed Todd also, could keep us here past wine and beer time. I have a couple of questions. I'll start with this one. We're talking about the evolution and you set up power structure and you kind of have labels to different levels. How do you check the people who are established in the power structure and make sure that they actually add value to this ecosystem and actually have a check to the checkers. Now part of the power structure thing the way we're getting around that is that we're going to have people like I say like Valtech, the rational, the IID, and PMI developers set, right? And check a learning objective. So the thing is that we want to alignment on is that these learning objectives are meaningful. So that's the first way. It's not we sit in the room, make it up and do it. So it's that double checking piece. So that's to get an initial set in place. After that, that's very interesting and I love it. We're all paranoid. You guys are even more paranoid than you think. But that's good. So who checks the checkers? I don't have more on that. Are you okay to let Todd get the next question? I know where you live. So... It's not like I'm near... So I guess I just found on the evolution how do we keep that gradual. And I guess the other element I'd be curious about, because I think that's what has been the biggest concern that my exposure to certification, that every time it's come up it's like, it's been, well, can you develop an actual certification program which isn't self-adjustable? And so that was one of the acceptance tests that I had in looking at what I accepted to something like that. I'm curious, have you come up with acceptance tests of what works for this program and user stories? Who are your stakeholders and user stories that you're concerned with? The kinds of people that we've dealt with first. Oh, first of all, I want to say that, as you guys know I'm on record, I think we're past agile, right? So agile is a word that amongst us we're kind of plastic. When I look at this certification you could strip out the word agile and make me, in a sense, happier. Effective, it's often development, right? So effective private management, effective testing, right? So when you say how do we keep this agile, I'm thinking, I don't even want it to be agile, I want it to be agile. This is software development people, not just agile. So how do we keep effective software development effective? Right, that's very good, and that's got the, I'll get back, I'll get, and so that gets to his thing about the other part of your question I wanted to get out, which was, what was the sentence? About user stories that accept this test. Yeah, so the driving stakeholders, so the thing that's interesting is, for me, is I don't do any work in the Middle East, right? He's got a fire under him and he's got clients who are learning from this stuff. I have clients who are hungry for it but not learning. So certain stakeholders are all of his constituency, there are companies in the Middle East who are trying to get their people trained. I have clients around the world, they come to me and say, here's one of the stakeholders, and this is the same as it was in 2007, Alistair, we've got 2,000 project managers who like this agile stuff. What's the curriculum for what they go learn? Don't tell me what course they're sending to. What should we get them to learn, right? So there are quite a lot of people who are companies who are asking that question, so there's a stakeholder in there. And then you take the opposite side, you get the future employee who says, if I want to get good jobs in this area and declare that I'm good at it, don't give me a course, what's the curriculum? So it's all curriculum based. So those are the stakeholders we've got at the moment. Do you have any more stakeholders? I mean that's sort of a huge curriculum. Yeah. Okay, I made it. There's some years ago I earned the... Yeah, I was stuck, I was very good. I missed that. The trainers, and there's 2 categories of trainers. There's the individual trainer like me and the big companies like Rational, Valtek and so on. And so trying to come up with it. And that's where the financing model, you know, cost comes in as you look at. Something that works for an individual trainer and what is it that works for a training and education company. Thank you for bringing that up. A dozen years ago I got a Microsoft Eagle Scout, the MCSD, and I've worked with it since. How do you prevent after-fee in the professional and expert level? Go ahead. Yeah, so the mechanisms we have in place is for the special professional, there's a renewal every 3 years. So again, if I can be discussed, it's going to be one year depending on the speed of the industry. And the renewal is through an interview, a phone interview, either face-to-face. Again, having that venue where people can meet face-to-face adds a lot of value because we can do a lot there. So either a phone interview, face-to-face interview, but it has to be talking to a human being, not just a tester. Another thing, we're now in areas where we have explored our huge amount, but Moana being a fan of date stamping and date that you get something. So if someone said CSM 2002, you know, they kind of haven't, or what if they said Agile Sir 2002 maybe they haven't kept up on scope. So these are kinds of mechanisms we haven't gone into. We have discussed that a bit, but I haven't settled on this, please. Yeah, I'm seeing in this and in previous slides mechanisms that I've noticed in a couple of different organizations I've been in. And I don't know what you want to talk offline later about some of the detail. Can you just name some of those organizations? Toastmasters, the Institute for Certification of Computing Professionals, those are the two big ones. And is there a similar difference to this? Similar in several ways. For the certification in particular, Toastmasters for the accredited speaker series. For the ICCA, they have the Certified Computing Professional. We've been having it for a few years. Is that Canadian? No. That was American. That came from about 20 years ago back when States were certifications. And are there any salient or really interesting differences that we might want to be able to archive of the Toastmasters and the ICCA? Or challenges independent politicians? If you can tell us later. Much more in details and I'm not sure how they would deal with the details, but it fits in and validates what you've been thinking here. So out of all this, I see nothing where you're off track as far as I can tell. And Lady can give us context as I would in my rechecking. How does the Toastmasters feel about what they've got and how does the ICCA feel? How do they feel about what they've got now having practiced it? Thank you. Back to you. Anybody else who has not asked a question yet? I'd love to see your first chance. Yes, please. I was wondering why you chose to do the re-certification every three years as opposed to what they do on the certified public accountants, which is peer-reviewed. And they do CPG, continued specialization. How do they do peer-reviewed? Do you know? I can ask my CPA because he'll know. And I'm hoping you'll remember some of these great roles. So not locked down, open the suggestion if it's a better idea. Cool. If there's pre-validated models for this that we can step into. The big thing is, as I said, we're trying to get, paint the big picture to get the individual certs that are out there in place, have it multi-stage. Even the playing field saw on. But so much of this is new for us. If we can borrow from other groups, it's just safer, better. Anybody else who's not asked a question? I know Jim. Look, Jim, I know. And he'll get you some more time, but people haven't asked the question right around all the disciplines. So you've talked about the focus track phase and the concept of, you know, like a lot of people creating their own courses to address the learning objectives. Does the focus track allow for other people to create their own merit badges that are still learning objective centered, maybe improved or, you know... It hasn't crossed my mind. Does that cross your mind yet? I don't see an objection to it. You know, if someone says, like, we really think this is very, very, very important and it's not in your list, I don't see an objection with having a, you know, sort of a plugging out on... The key is, like I said, open, additive and just transparent to get the job done. Okay, back to you, Jim. All right. So, these things never work out the way people do. So, but maybe it'll work out better. Yeah. I have seen we're going to get sniped from some place we can't even see. Yeah, so that's my question. Is in five years or however many... some time in the future, substantial amount of time to actually see results from this, how do you evaluate whether or not this is working and what mechanisms do you have for pulling the plug if it's actually being... So, I'm a big cynic when it comes to human nature and let's say that I really disagree with it and what I would then do is of course quit. And all the people who like it will of course not quit and then I'll go out and say I quit and I don't like it and they'll say stuff you and keep going. That's the kind of... You try your best to keep it, but I mean... Those of you who know me, you know, you must know I'm terrified by the entire concept of... the entire power centers all by yourself. Right? Only for him, he's built that way. I'm not. This is like scaring the... And one other thing... But... We're hurting badly without it. We're hurting really badly without it as someone here said, this is the best plan that I've seen. That's actually me. That's not bad. That's how nervous I am. May I make a suggestion which is that you incorporate a kill switch? Nice. I'm assuming that your stuff's written down which means that you can forward those stuff. I will email. I suspect you will. In fact, you could blog about it and then people could comment on your blog and you can see the comments on the blogs and the evolution there. The thing that's really pleased me, note that I'm the add-on to Avnet, right? In a sense, we did some foundational work. We came back in 2004, five, six, seven. Right? But he picked it up and took it to the next step. He came back to me in the night and time. I'm signed on for the ride. And the thing... One of the things that impresses me continually about him is open, transparent, right? We're telling you everything we know. We've got publishing objectives. We'll publish the tracks. Well, everything is open. The only way that we can... That feels safe about how we're doing this is the bank account. Is the bank account open? The bank account is open. We've got deposit slips. But that's the way. Because if we have a problem around some of this, the only way that we can see to do it is to get as much open as possible. And one other thing, this has evolved since the flight. So the fact that we've been trying this out for six months, a lot of things have changed. Right? I think you'll find it much harder to find a lot of insurance. Yeah, yeah. Let's get the next person with... I'm having... Sorry? Ali? Did you have all? No. One of the reasons we're hurting is because of the existing certifications out there. All the reasons are really hurting. To hear the thing, without any certifications, I go to some of my major clients and then with the problem I said, we've got a thousand project managers, we've got a lot of saddler stuff. What do we go send them to? Don't tell me a company, don't tell me a course. What do I have more? I've been begging for an answer, but that was the thing that drove me at the time of the APL. I'm looking at what was driving me at the time was I had clients who said, give me a learning structure. That's for me, the burning platform. That's the burning thing. You can't function in his venues without it. So is this specific to certain clients in certain cultures? You've got the alcohol thing around the CSM, which is people around the world are just getting mad. I was going to say, I trained here too. Some of my clients here, first thing they asked, is there a certificate after there's training? Is it part of the certification program? Trust me, the culture is not just overseas. There's that culture also in corporate America a lot. And the question is, I guess, what the SCROM certification, the SCROM Alliance did was it provided one alternative. I think that really pushed just agile forward in spreading it. Now, the quality of it is a different discussion to talk about, but it gave the mechanisms for organizations and enterprises to say, okay, give me these certified things, whatever they are. Because there is something out there. If there isn't what alternative do people have? They learn by themselves. I would love that, but reality is a different discussion. Can we just have a discussion? Yeah, go ahead. So the reality of you talking about the pain you just mentioned, the pain is you guys can't sell. I know. Just throw it out there. Listen to my words. I go to a client, they say, Alistair, we have a thousand PMs learned. Do not tell us a course. Do not tell us a company. What should they learn? When they start sending the people through, how do they know they've gone through it? So that is a pain. It's not me. I've got nothing to sell. I've been on the other side of this where a community is begging for a certification. It's almost always I think it's going to lose you. These command and control structures, they insist on having the rank or the mirror match at the end of the process. I'm a cynic on human nature. I think that is a given. What he said, give me five of them, certain things. We know that's going to happen. I'm coming from a slightly different take on it and I'm seeing folks that are going for certificates or be equivalent as a self-actualization, a validation of their own phone ongoing growth and skills that they know that they will apply. Whether or not it's an actual recognition. It's cool that it's such a minority. Maybe I'm in the wrong spot. No. That's definitely not a majority. And we'll get to this in a second. We got a little stacked up, but just one thing about that. I forgot. Hang on. Let me add to what you were saying. When you look at reality, we have one or two options. Either to say, no, we're not going to do this. There are still people that are doing certifications that are of different quality. So let's talk frankly. Let me say one thing. There's basically a power vacuum that's being created by the disintegration of the scrum like... Actually, the opposite is true. There's not a power vacuum there. There's actually a power monopoly right there. And I would say a knowledge vacuum. There is a knowledge vacuum, but I think if you don't think there's a scrum backlash, this is a scrum backlash. It's different than the previous energies. The vacuum is forming. There's not a vacuum... The scrum line still has power. There's still quite a bit of scrum certification. Is the vacuum certification... They just split like an amoeba. But let's be open. We're going to talk. That's dysfunctional. That was a dysfunctional organism. Right? On whatever level. And that power structure is not as strong as it was. And that is creating an opportunity for someone, maybe you, maybe someone else to create a certification that people will value as the agile certification measures what scrum has been for the last however long. Yes and no. There are actually a large number of people showing up for certification. The power vacuum is not there because the scrum lines and the CSM still has the power. However, there's a vacuum as you identify and we're seeing happening. Right now, we're all competing small certification things and we're trying to make a big picture to get these things to line up. And even just putting the whole vacuum thing aside. Quality. That's just a whole different discussion by itself. That the quality of people that are certified today we need to step this up and give them a path to even create a journey home. So we'll add to that. Daniel's spiked up and he hasn't spoken. Sorry. There's obviously a need there. I'm not a big fan of the Austrian obvious. If there's a need, the market's going to support that and it will eventually survive. If the need goes away or whatever that's going to collapse and go away. So as long as there's that need this will be successful and continue on. And this claimer here I don't think any one of us claims this is perfection. But it's a step forward. Okay, we're out and free. The DSMH terms have taken the ABL and stuff and created a natural project leader curriculum thing. Not that much. And to my knowledge it hasn't been particularly successful. I'm curious as to why you didn't go back and look at it. Why not? This time it's just bracketed. This is part of the basic due diligence. Say, same questions. What do you have there? It's not that different. The biggest thing that's different is the hands-on merchant piece which sounds intriguing but also sounds expensive. So I'm just curious have you really thought that through in terms of how we've done that. I don't have the numbers in my head. Yes, it might be expensive. The idea is if you had let's say 16 or 20 people going through, they split the cost and put some show up in the movie games and we'll be assessing. If you get a little bit creative about it, you can charge admission for the visitors. Come watch Agile People program 20 bucks for the day or something. We have somebody who has a phone before. Please, Jenny. What you were talking about is really what we do at Backcountry. I want to clarify it's not successful in the U.S. It's very successful. No, no, no. DSTM is one thing. We're talking about the certification that was added. I think the certification as well is being adopted over there. I looked through it a bit and I find it interesting. It might be worth just looking at it. It's part of the due diligence. Our target timeline, by the way, is to do a proper announcement at Agile Conference middle of August. We've got two months. There's hurdles, you know, financial, legal. And this kind of due diligence reaching out, checking for people. Please, more feedback. Even after, because I know our time is almost up. Ali also hasn't spoken? I don't know if anybody here talks about certification. It's a hands-on. You get to group of people they have the same interest and they try to learn something every day. I took the Scrum Master and College Honor certification two days after the $2,500 that came out. I did not learn as much so if quality adds to this I see it's going to get successful. Okay, I'll let you and Jim think it out before you get to the next term. Oh, wait. We have a new voice, Nate. With all his merit badges am I going to get a nice sash so I can put those back? A t-shirt store? A hat! And we're going to get into the store different qualities of cloth, different sizes, different prices. Talking to Disney right now. You presented a post-acrony, I think which is that either we do this or they do it but I think there's a third option which is that it is possible for us to debunk the idea of certifications as a valid approach entirely and right now if you Google as a certification you'll find my article on trying to do that as a third hit so it is possible. So I offer all of those paths at the same time. So I encourage you to debunk certification at the same time you're building it up and either you will convince the various other people like if you were to totally convince our clientele to go away I actually think that it wouldn't bother us the whole lot, right? You know what I'm saying? I'm convincing to go away then we've got this model on there so I actually don't want to dissuade you but I think that's a huge mission because that's culture. So right now I'm the president of the Downtowners Toastmasters Club Are you serious? Yes. As we talk about the quality so there's a couple things that came out that he brought out originally the Toastmasters you have this notion of a community of practice that everyone's giving speeches and you go through and you deliver these format speeches. The thing that bothers me about what we're seeing right now a little bit is this notion of interviews as the gaining factor there's still got to be some qualitative notions because we're human beings and I don't think you can ever get away from that but I'd like to see if we value things agilely and you go back to an FS so the value is in working software so I'd like to see something here in the certification that actually is working software just interviews teacher, teacher how do you do that for a business analyst project manager tester there needs to be something that's right here that's too late that's not the only place to be seen remember now we have a knowledge certification we did not mention certification in the first two phases this is a path you can't call something an associate and not tell me it's a certificate what? it doesn't make sense it's not a certified certificate it's not a certified certificate come on you don't have to be an excellent person we're about at the end I just didn't understand your point now you know where I live so you can come get me but for the audience and the camera can you elaborate on what a deliverable I don't say working software say working product what would a project manager or a business analyst if you're a project manager and you're a business analyst and you're delivering something the thing that comes in my mind is something like a story map or something that's a consumable in the process that you're actually going to participate in all ideas all promote I would propose an experience report like a portfolio something you've actually done that's an actual interview that's supposed to be about but actually this is a portfolio thing and you've gone back as to what that would look like the other thing that concerns me about the hands on immersion it's different than a toast master who's doing what they actually do to give them a speech in this case you got hands on immersion and the team that may not have ever worked together before and storming and all that stuff it's got to deal with it's development his point which when I raised a certain objection about that he said come on Alistair four hours you don't think you can tell if this guy can do some initial programming even if they're in a dysfunctional you can spot the ideas that's the point of having the experts but you can't necessarily determine whether they can produce working software whereas in the experience report they have to be good liars and that's where the idea that he's got is get a real honest to goodness customer a real person from a business to come and say I need this from my business and then you see the code getting built you heard that he did Pat Maddox was talking about one day the real software is being built actually in that day the one I've got concern about is why it's from master to man get a complexity that's actually the role that I find most difficult to assess the other one is the business analyst the programmer tester, UI designer you can see how they do their stuff and the real stuff last question a more comment if there was opportunity in the focus track opportunity to do some portion of what would end up being the hands-on immersion of the certificate phase have that built in as a possibility in the focus track phase I think that would be just to follow that remember each one of these phases has assessment so I don't expect that the development track will assess through an exam and that's part of where if you propose that kind of assessment we'll probably say no do some show part of your assessment so out of time obviously we'll be giving them a long conversation in case they're out of reach thank you