 All right, so hopefully that kind of gives some light into why JP Morgan is kind of interested in this conference And you know, we would love to hear more from other companies as well who are going through this transformation We do this public submission system where we try and get as many speakers locally internationally to share this story next what I want to do switching a little bit of gears is kind of We've we had actually a keynote speaker today, which was Jim high Smith. Sorry Jim high Smith was not a keynote speaker. It was Jim McCarthy. Who was a keynote speaker? Jim McCarthy could not get a visa to India. So his trip was cancelled last minute But we thought we'll take this opportunity to bring some of our other speakers on stage and do You know like a panel where people can ask questions and each panelist will get a chance to take a stab at it We'll I'll keep a strict time on each question and I'll time out For each question not more than five minutes. Every speaker will get a chance whoever wants to so let's welcome Arlo Bell. She Ravi Ravi Kumar Ravi's the team chair We have doc and Chad all of these guys have worked with a lot of distributed teams and In my opinion have a lot of great insights in in you know How distributed teams work and how agile teams in general work? So I would you know be delighted to hear their opinions of things that could not get addressed today things that you still have in mind So anyone wants to take a stab. We did have a theme for the panel Which was more of you know offshoring and agile, you know, are they kind of oxymoron are they really? You know that can they work together? That's really the theme, but we will see how it goes and also if When someone puts up the hand and ask a question if that you know if you feel that the speakers could not answer that well Then we welcome you to take one of the chairs And you anybody from you can come pick up that chair and answer the question All right, and I would love to see those two chairs filled We will use a fishbowl format. I'll just quickly explain what a fishbowl format is So these guys are fixed we put glue on their seats and they're stuck They cannot get up, but those two chairs don't have glue on it So when we have someone from the audience who wants to Give an answer to the question you come grab a seat Someone else wants to give an answer can also come grab another seat But the rule of a fishbowl is one of the chairs have to be empty So if the second person comes and sit then the first person has to leave In terms of after answering the question after answering the question. All right, is it clear if not we'll figure it out So who wants to be the lucky one who wants to get the first question right there go Can we have the mic please? It's cost reduction if my primary intent of distributing or offshoring Will Agile work there? That's something that I'm trying to understand So let me repeat the question So if cost reduction is the primary factor for distributing the project for offshoring the project Will Agile work in that kind of a context? Amler has some good data about this from several years back. He was looking at Distributed teams versus co-located teams and comparing that to Agile and non Agile So there are four possible combinations And it's worth pointing out. He was looking at adaptive iterative not just Agile So nuance that doesn't make much difference here, but but includes all the Agile ones So in that mechanism or in that Organization you found a fairly strong swerve in terms of productivity in terms of how much they would deliver for a particular unit of cost He found that co-located Agile teams had the highest productivity And then below that at some ratio, I'll just say 80% I don't know what the number is is co-located traditional teams And then there was a huge gap and then there were distributed Agile teams and then distributed traditional teams And so if you are distributing anyway agility is still a benefit However, co-locating has more of an effect than switching from traditional to Agile in terms of productivity Yeah, go ahead. So I Distribute for my cost reduction because my cost is less there and then I go back to the Off-show team and tell them that You know, I've started just you know giving you work now get me better productivity because I still want to reduce the cost And I don't want to invest on you know I don't want to increase my cost out there because I'll get I want to get more value from that The center, you know, how do we tackle that kind of a situation? They want more value, but they're resistant to cost How do you tackle that? So we'll pause on that question We'll let the the speakers respond to your first question Which is you know, uh, you know if if cost is the factor, would you go distributed and distributed agile specifically? So What I think is What what I think is typical is there is a cost productivity trade-off, right? So what do you mean by cost, right? Is it hard dollars because the productivity? Hit that you take actually is a cost as well and sometimes it is worse for you In terms of if you would do a team that's co-located and together Versus what the productivity hit was now we do a lot of distributed agile projects And we found ways to make it work and we actually help our customers do it But some of the things we find is can you Take the distribution and still make it kind of co-located like so you're talking about how you could empower those teams Well, is there more work that they can do together as a unit versus You know 50 percent of people being over here 50 percent over here and they have to figure out how to collaborate So I think it it the parameters Um Really matter in terms of you know what the cost outlay is how your teams are working to get that The proper benefits that are That makes sense I think Cost yes, you can reduce it But so long as you have certainty in what you are expecting So if you have got requirement, there is certainty and you want to drive low on cost Yes, you can outsource it or Distributed and get that But if something is on an emergent nature you are trying it out and then expect Cost also to decrease while you are distributing then I don't think it would work out They don't go linearly in that way So you'll have to know what is it that you are distributing and why is it that you are distributing and what's the outcome that you are expecting So that would be the fundamental basis. Why would you want to try that? Hey, hi My question is In your own experience What kind of fixed price contract models have you seen succeeding in an agile outsourced projects? What kind of fixed price agile outsourcing contracts you have seen work Essentially the key factors on which this fixed price contracts were built which you saw being successful Especially in the outsource model Yeah, so You know the projects that i've done where it was, you know fixed cost and and and we were providing You know the actual staffing to do the work Really the way that you know, we made it to be successful was um focusing on value focusing on Um, you know being able to to flex the scope of delivery So we can we can guarantee that we're going to have something of high value by that timeline and with it You know within cost, but that does mean that you know, we've got to be a lot more Open about the scope and we've got to be disciplined about making sure that what we're building is You know high value stuff and you know that we're not over architecting etc And in getting something out there fast and iterative So, you know if the if the cost is fixed and the timeline is fixed then scopes got flexed in some way To me Fixed price is an evil It's a crime You cannot be doing that. I mean in the in the sense that how it is being Distributed or offshore today because everything it's from a cost perspective You want to drive it and Often times not often time most of the time in my experience. I've seen it's always been the iron triangle It's always going to be fixed fast fixed scope and your fixed quality etc. No, there are there has to be a trade off And the moment you see fixed price and trying to drive down that aspect somewhere There's a waterfall trap because vendors are also going to hit back and say okay Give me something of certain then only I'm going to commit to this So then you are killing agility You can still be running your project under the banner of a jail, but I don't think truly you will be a child And just a quick additional point. I think rarely is scope actually fixed in any way So When you get into a space where the scope is fixed say there's a Master story list that you've agreed upon delivering rarely is that really the list And that'd be a good thing from an agile perspective, right? Let's deliver the highest value first, but it's like oh, but what's in the contract is this master story list So then you end up trying to deliver the value and the list Which that's a trap as well. So I think if you're going to do a fixed Price thing you have to be very careful about the prioritization parameters and the scope parameters in particular because It usually up front. You don't know what you're going to build. That's the point It's it's all about the feedback loops and and being able to use those to maximize value That's the reason we do this whole agility thing Feedback's only useful if you integrate it into what you're going to do. Otherwise, it's just hopes and dreams and wishes So to include that you want to give yourself options in the contract Toyota's variable scope contracting is a great mechanism for that It does a good job of lining so that both teams win if scope is decreased So, you know, there are techniques like that that you can write into the contract so that you can incorporate future feedback and learning And I recommend pitching it that way as we want to include learnings that we gain over the project in order to drive the costs down and get more value Yeah, there's a lot of organizations have multiple divisions, right and and You know, you've got r&d and you've got production And I think that one of the challenges that we face one of the things that one of the changes that we could Look at making that would really help is we look at software development as r&d as opposed to as production And if funding were done in that way, I think it would change a lot of this a lot of these questions, right? It would change the context of the entire conversation And I think that's a much more appropriate paradigm for software development Thanks Go go there. No one from the audience wants to add anything in Is there's two seats still left open? So there was a time when xp was I think we need to just hold on to your question. That gentleman has been waiting for a while. We'll come next to you. Sorry evening all uh My question is related to release planning Uh, often I mean I come across project where I'm I'm given a prd document and uh, people say, I mean They just say that, you know, I'm okay with whatever mechanism or Approach you want to take to execute this but I need a date by which I can come You know communicate back to the client or the customer as to you know, let's say 30th of May you're going to get this How do we go about this because if you if you want to arrive at a date then we are going to calculate velocity and then you know in In a way effort and then days and months and that that kind of thing. So Is there is there a silver bullet to you know, actually Do that So I think um From my personal experience, how much is the certainty that you have on hand To actually go back to become a customer and say that I can give you by this date if you ask for a date If you don't have certainty, then you obviously cannot commit for it At a at a high level. Yes, you can and that also is a high level expectation from them. That that is that is okay Uh, they would want to go reprogressing it in that direction and if it's on an emergent evolution kind of a thing Yes, it's perfectly valid to be there and you can benchmark against that and progress but the point is uh, if you are Giving me a high degree of certainty and then expect driving it then again if you are getting into the trap So it's not it's again killing back going back on your agility In my mind, there is a silver bullet. Um, and that silver bullet is data data data and more data So, uh, there are teams that make estimates by asking people for numbers of how many days it's going to take That's a fairly data free way to make estimates And you don't have a high degree of predictability In your results the teams that do it by Just counting the number of stories or maybe estimating the their complexity in a unit and then measuring a velocity Tracking that over time checking for your variance and velocity as well as the means using that to predict measuring the The incoming rate and outgoing rate of stories Outgoing being stories you choose not to implement When you start incorporating that sort of data into uh your analysis of of what's going on in the project Then you can start making very good predictions. And in fact, you can start making predictive models that I can then Uh rather than me making prediction for you I will hand you the model and say go ahead and adjust all the backlogs However, you want freely in this model and it will continue to make predictions of What you're likely to get given those decisions and then pick the right decisions So data Two points. One is uh, I think you can make escalating commitments, right? You can say At this point, I can give you this broad brush stroke with this level of certainty So you can create range of date and certainty And as you progress with data You you can tighten up the bounds of that second thing that I find really useful is understanding cycle time So if you understand From soup to nuts, okay This is a story these are how long it takes me to do this from soup to nuts This is how much work and process I typically carry That I can carry that's good for my team You can kind of look at it and make a very good guess from there because you know, okay My cycle time is 15 days and I haven't started it So there's no way I'm going to do it in 13 Right. So I think that's a very useful mechanism to start to to make those choices Yeah, yeah, absolutely. I agree So, you know when I I'm working with teams a lot of times we go in and and some will say We've got to get this team to be faster, right? Um, and a lot of times the first step is to actually slow them down Um, step back take a look at cycle time Take a look at lead time take a look at, you know, all of the data that you can actually gather from that team And figure out what the root causes are to actually stabilize the throughput once you've got that predictability Then you can actually start making, you know, some level of commitment with certainty You know, we'll do burns with standard deviation rather than just a standard burn So it shows what the range is of, you know, our confidence interval against that But you know, if the team doesn't isn't isn't stabilized first One you can't actually get them to go faster because it's There's there's too much gunk in the works and you can't make those kinds of of commitments not not with any kind of certainty So I'll take a stab at that question as well. I'll jump in from So I think this is the first I think Jeff Patton was kind of explaining this model and I thought it was pretty interesting So he talked about the you know, trying to look at the iron triangle And he said if you look at the iron triangle, you have basically your scope your cost and your time And typically people would use Uh, you know, this is fixed the the scope is fixed, right? And then you would use uh estimation to figure out what would be the cost and how long it will take Right and then he talks about in the traditional models. This, you know, is kind of what we focused on But if you look at, uh, you know, what some of the agile methods are trying to do is they're actually trying to invert this Triangle and they're trying to say well, let's fix the Cost and the time In in other words, you you give me a deadline by when something needs to be delivered, right? And we'll fix the cost because we We have a team of six people working on this or a fixed size of team working on it So we know the cost is fixed for that duration So the cost and time is basically fixed and then you basically estimate the scope what that means is you you you Work in a short cycle You work at regular heartbeat and then you see measure how much you're able to achieve in that short duration And then do constant projection going forward to estimate what scope you will be able to cover in the given time Or in other words, you're kind of trying to uh, also budget things saying, you know This is how much i'm willing to invest on this kind of a thing and then, you know We will play around with the levels of sophistication How sophisticated something will be or how less sophisticated it will be depending on your budget So that's uh, in my opinion, that's kind of a very different perspective of looking at, you know, this this whole iron triangle problem All right, we'll go down there So there was a time when xp was uh, quite popular Extreme programming is one of the agile flavors. Um, at least I haven't heard about that being practiced anywhere Hello, sorry. So there was a time when um xp was quite popular as one of the agile flavors. I mean probably couple of years back Now off late. I haven't heard any of the organizations at least to the best of my knowledge practicing xp So in your experience, have you seen xp being practiced still or is it that scrum or others are kind of defective for practicing? So, uh, as a strongly xp biased person I'm going to answer accordingly. Um, so scrum has a number of very strong advantages over xp. Um, it is marketable Um, it is easily transmittable Someone who has done scrum for a short period of time Is able to transmit that knowledge to somebody else And therefore it has a much shorter doubling time in the marketplace So it's not that xp has decreased the amount of xp in use is actually increased But is increased with a very slow doubling period. It takes a very long time to to spread scrum has been driving all of the agile adoptions for a while because it's marketed well well And it's easy to transmit and so as we start getting more and more agile visibility. It's really a lot more scrum visibility But we are still finding that The teams that are really delivering well If you want to get to two star fluency xp is still the game that that gets you there Scrum is a great way to get to one star fluency We see a lot of teams get there and then fall into certain technical debt traps and then go look for xp So one of the things that i've that i've seen happening is in a way scrum is consuming xp You know, so i was involved in Creating some of the scrum certified developer curriculum and teaching it a couple years back And it was a scrum course and it was a scrum certification, but Throughout that course we were talking about pair programming You know test driven development continuous integration You know, we were talking about all of these you know, many of these practices that actually came from You know xp And just kind of folding them into You know into scrum training. So it's a little convoluted these days a lot more than it was I would say You know four or five years ago I think there's a bifurcation of people thinking about scrum in terms of project management and analysis and There's probably an overlap to the extent that xp did talk about stories and using stories and pushing those through But that now when you hear people say i'm doing engineering practices as the the folks Forget what company that was up here Yesterday said, you know, we're moving from doing all scrums and now we're doing engineering practices. Essentially they're adding xp So people people might not be saying that they're adding xp And the word also became dirty for a little while because people said extreme programming Why did you name it that so now people say i'm doing agile engineering practices. They're doing xp So xp is is alive and well and flourishing I've actually not been to a single organization or a team which is doing code in code scrum And is not doing mini waterfalls I have not seen a single company in 12 30 years of my You know experience with agile who's doing code in code scrum and is not actually practicing mini waterfalls This is essentially mini waterfalls every month Or two weeks if they can sustain it one month itself is hard Two weeks is burning people out All right We'll go Yeah, there's a gentleman who's already got the mic and we'll go there next Good evening everybody. My question is this is going to sound a little bit skeptical Last two days would be my first exposure to agile and hearing lots about agile And I've been hearing nothing but um, almost like a nirvana state, right? Everything is going to be great customer gets early We are going to have less surprise and low cost self-organizing and it list goes on My question is if this is so true and is and and I believe so based on what I heard What stops some of our executives, you know, I assume all the ct os and c o's are the best in that Why isn't that um, there is a big of jump into this bandwagon and we see almost agile development across the board Versus what we are seeing today just more waterfall and then less of agile So I would like to hear a little bit more on what do you think is some of the blockers from the executive standpoint? Are there some of the blockers for adoption of agile? So I think that this is a highly culturally nuanced question So the blockers you're going to see in the us are totally different than the blockers you're going to see in india And the rate of adoption that you're seeing in in the cultures are going to be entirely different Like in the us at this point people are talking about yeah, it's pretty much everyone's doing agile Only the laggards are left and they're already flipping The waterfall traditional stuff is mostly dead And the challenge is that a lot of at this point the transitions are in name only They're doing the same things they were doing under waterfall which weren't waterfall anyway And now they're calling it Agile just as they were previously calling it waterfall. It's really cowboy So it's it's not that people are You know are avoiding the agile transitions. It's that they're we're down to the point that it's the people who are white labeling Whatever it is that they're doing anyway That's not that's totally different when you go to different cultures One of the big things that I think is is a driving part of that is Agile done well fundamentally decentralizes power Uh it fundamentally shifts from central decision making to much more distributed decision making And if status is associated with who makes what decisions Then that is a shift of status power and influence all at the same time And that has huge cultural ramifications In companies and uh creates quite a lot of resistance and back in the the days when agile wasn't the dominant We very often saw agile show up in one or two portions of the company And then the antibodies would come back and come out and destroy it. It was because it was decentralizing that power So to me, I mean piggybacking on what Arlo said um My experience has been I I've worked some time in the u.s Uh while I'm from India I worked some time in the u.s And that cultural thing what manifests in the organization makes a huge difference We never had agile as in agile as a banner, but then there was always customer collaboration working with the business Uh, I had not read about agile at that point in time, but I remember The manager making a statement if something goes down in production, then I'll get to know I'll come and talk to you Otherwise business will not complain to me. You are doing a good job Right, so that was a perspective coming back to India there was a whole lot of process and all those things and The experience was that we never did a perfect waterfall In India while we talked so much about waterfall and things like that. We never did perfect waterfall Because it was always chaotic Requirements was never frozen. In fact, we got to know the requirements only in uit The point is when you go to a waterfall and it's kind of I have I think stopped making a comparison between waterfall and agile That way but then the good aspect of agile what has to be taken in or inculcated is the the value for the system and what Arlow was mentioning the decentralization aspect You cannot be chaotic and then say that is agile Which is again a misnomer there. It requires high discipline to be agile And it also requires some of the authority and etc The powers to be delegated back to the teams and they can govern themselves So that takes a fundamental shift and we are not in that kind of culture if I have to talk We are more control oriented as opposed to giving off that so I'm going to say something that might be slightly offensive, but I'll say it anyway You have to just be yourself Jared So I think there's actually what I call a generational lag in technology management And that is this generation of managers was last generation of technologies developers Right that makes sense right and so Oftentimes you have people who are managing now who are disconnected from What's the current state of practice both in technology and in process and When you have to explain to someone who you know Cracked their their life on cobalt mainframes and their worldview is developing on that And then you're saying pairing unit testing all these things that are kind of foreign It's hard to make the intuitively that all these things are wonderful and I should do them Right. I had a conversation with a business leader about continuous integration once many years ago And we went through this whole thing about continuous integration and why it was wonderful And she said at the end. Okay. Yeah. Yeah I'm just trying to make my merge go from eight weeks to four weeks So how can you help me with that? And I said but there would be no merge and she's like, I don't believe that that's crazy talk I'm like, no, no, no people do this all over the world And she was like, you got to be kidding me This make I want four week merge forget that see eyes. I don't believe And I think that's um, it's hard for people to crack Yeah, so uh, I'll we because we have just started our agile transformation a year back And we are a 260,000 people organization and thousands of people in technology So the first thing I think that the organization has to believe that that is pain We we are in a banks are in a highly regulated environment Regulations are changing almost every day And we realize that our technology is not agile There was another problem to to make it Make the organization agile what we realized it that most of the people They stopped learning the moment they joined the bank. Basically we we learned through our college But once we join a company That's when we stop learning that is we don't pick up those books Well, there are of course passionate technologists like the folks sitting here But mostly in most companies people don't learn So what we have decided is that we first want to enable a learning organization And and agile is very very difficult to get it right, especially in a large organization for a small company like whatsapp It's very easy. Uh, I have worked for a startup for 10 years in the u.s. So I know but but here The to implement agile first things is those managers Have to go and sit with the teams and work with the teams in implementing those best practices Well, I would even say it as good practices because you evolve them And uh to to just tell a team to implement continuous integration It's not going to work to make them believe that continuous integration is like breathing air It takes time and it's going to take a long time For any organization that starts with so we have started at the sea level first So when we hired this World famous coach in lean agile the first thing he said is I am not going to speak to anybody but the sea level We are going to first spend time a lot of time with the sea levels We are going to coach them and then we'll go Down the organization. So It has to start the coaching has to start started the top not at the bottom Yeah, so The reason I came over here was because I went through that transition from, you know, doing sdlc to agile and I've also spoken on that So typically we what we've noticed is especially if you are in a service provider setup In many cases, uh, the customer would say we want to go agile But why they want to go agile? And what agile actually requires are two different things. They want to go agile because they want things faster And and in this case when I say customer it could be the management, but they don't realize that they need to change as well So it's it's like we need to go agile. I need things in two weeks And you guys go and do it. So that's the first problem Expecting that we don't need to change You know, you guys change and then start and the second thing is investment So there is a process side. It's sometimes easy, you know, you can start stand-ups What's a big deal? People just stand up. You can do a retros. What's a big deal? You know, give an hour do it but True transformation happens when you when you have to invest in the platform, which is, you know, continuous integration Automation test maybe performance test. You actually need to pay money to get licenses in these cases. You probably need to get hardware People don't think about it. You don't actually put in that initial investment to make those transformations So you expect with a stand-up and a retro for the teams to just transform and start doing things You expect that teams will do showcases With just a stand-up and a retro being instituted that doesn't work That's been the typical challenge that we always have All right I want to add to that basically I have two perspectives, right? So you were saying what is stopping companies from jumping on this brand wagon? I see every company jumping on this brand wagon Tell me a company which is not jumping on the brand wagon I can't think of any. I mean I've spoken to so many companies. Everyone's jump on the brand wagon A lot of them are dropping off the brand wagon as well It's it's 40 years now. And I think it's pretty Old in my opinion If you look at a lot of Indian services company, they are jumping on this brand wagon for one specific reason Any guesses? No Customer required it if I do it. I get business if I don't do it. I don't get business I did cmm before this. I did something else before this. I'll do now agile, right? Send 10 people get csm certified send 10 people get some other certified show that as the Profiles we have in the proposal that we send and there we get a project, right? So a lot of services organizations in my opinion are falling into that trap, right? So I think to your question. I am not aware of a single organization I'm sure there are but I'm not aware of a single organization. Which is not jumped on this brand wagon right I'm talking about services organization now. Let's look at India has a huge number of captive product companies, right? So you have a product company in you know in europe or in us or in australia And then they have an offshore development center or an odc over here, uh, you know And why are those guys jumping on the brand wagon? Because they want to bring more control over here They want to have better say in how things are being done, right? So that's another direct motivation for people to kind of Show that we have agile skills and trying to get better work instead of just doing glorified generator's job Right, so there is that other factor for people trying to do Bring a lot of agile into the organization. So in my opinion, I don't see what you're saying as Happening at all. I see the exact opposite happening I don't have the numbers if top of my head. I would have to say 90 percent of organizations claim to be doing agile 90 percent of organizations claim to be doing agile Doing agile itself is an oxymoron in my opinion. Uh, if you really want to go more into it, I would strongly recommend watching Andy Hunt's webinar that we did before the conference where he talks about, you know, the day we wrote the manifesto It was dead for him Because he says that, you know, it was essentially We try to make this as a static thing while it was something dynamic heading somewhere that we didn't actually know And we've kind of made it very static now So if you want to see the numbers, um There are two reports state of agile report and state of scrum report And they have a whole bunch of data on who's doing what and how many Uh of all it projects are are in there or whatever. I don't have the numbers off the top of my fingers But lots of people at this point are at least claiming to to do those practices Right and there's a lot of fudge for example Historically continuous integration on those reports is like 50 percent When you dig in into it with people and you say are you doing continuous integration? Yeah But my nightly build is running just fine No, that's still better, right? I have the ci server sitting there Right. We are doing continuous integration because the server is there Right. How often do you check in? Uh Once before the end of the sprint Right That I'll be lucky. Yes, we'll go there Hey guys the so it's quite clear that agile is certainly booming. We now have frameworks to scale agile Thanks for the sarcasm So, uh, my question is uh, what's the panel's thoughts on the scaled agile framework? You want a public answer to this question? You're outing us here So, um So, okay, first I'm gonna gonna Again state my biases. Uh, so I work at microsoft. I do uh company transitions all over the place. Um, You know, I when I do a pilot a pilot is 150 people at a time. That's just a small thing you try on the edge Right. Um, so when I'm talking about scale, I'm thinking large scale Um, so I'm not going to propose an alternative Way to to handle scale until it works at least 600 700 people at a time Um, from the conversations that I've had with Alan shallow way on scaled out of a framework It's really well targeted at the 150 person size for which I say well, that's interesting. It's a pretty good pilot Let's see whether it works now. Go ahead and see whether it works. I'm not going to bring it in in my case Um, there are some things that it says Are important, but I don't feel this is enough emphasis on those and in particular The technical practices If you're doing the technical practices if you're doing xp like or adaptive engineering at those teams You just drive a lot of risk and cost out And when you drive out a lot of that risk and cost then you have fewer problems to solve at the higher level There's a whole lot of weight in that in that framework that's There to prevent Screw up cascades to prevent an error in one team from cascading to another and another and another Um, if you're doing the technical practices You just don't have as many screw ups and each team has more ability to recover from a dependency screwing up So you don't need to prevent those cascades as as much and it makes me wonder why you have so much Heavyweight process at the top for dealing with what can be a non problem if you simply solve it to me I I I cannot give a direct answer as I log here because my work is in a different kind of an organization It's predominantly it services. I really don't see a point of scaled frameworks in an it service organization We are so dynamic There are multitude of complexities multitude of service lines and industries that you support I really don't see how exactly can you scale other than the verbates that you go into carry with you Because fundamentally context matter and these contexts are so much of variability in those contexts And when you look at scale and things like that, there has to be a certain amount of standardization vocabulary and those kind of things all put together And commitment from the management and also the customer There are too many ants and I see lots of ifs and ifs says if they work then it happens I don't see the first if happening So I don't see how it would actually do that We can turn off the video I think obviously there's a need to give people And I said this in my process talk about an hour ago guideposts on thinking about how to think through things But I'm I'm suspicious of any Framework in general that says hey to be successful you should work like me because 10 other people have done it So I'm biased there and I'll put that out There are a couple other issues one is I think the skilled azure framework doesn't say enough of what not to do So there's a lot of stuff in the framework to do this do this There's some heavyweight stuff in there, but there's very little don't do this And my what I've seen observed is that organizations with bad practices And bad ways of thinking that want to scale agile can wiggle bad behavior into lots of the little Nooks and crannies of the framework, right? So you'll see people do things like analysis decomposition Which the framework talks about in a very nice way about you know at this level do this at this level do this But then you see people where essentially at some high level of the Organization the analysis is what's law and then it's being pushed down and shoved to teams And at no point do people say this is bad Don't do this kind of thing as part of The framework itself. It's so heavy on evangelization at this point Do this get out of there is great that people aren't saying do the stale azure framework, but don't do these bad things The other bit that Concerns me is I think there's some There's some voodoo math And how certain things are set up so the normalized estimates across teams It's just I look at it and go First of all, you can't normalize estimates across team getting people to estimate the same way Is a fool's errand in my Opinion and even if you could The point of it in the framework is to help you make ROI Judgments about what to do next That would assume that not only the estimation itself is consistent, but the execution Is consistent So if I estimate something as a 30 the other people estimate something as a 30 We both on average have to do it as a 30 for it to make sense at the ROI level But if you do it as a 15 consistently and I do it as a 45 consistently Well, then Did my estimates mean anything? So I think you know and you see this too with people doing Scrum at scale and rolling up release Burndowns across story points of many teams. There's just lots of stuff where you're like Man, did anybody really think about the math of that? So I think that there's some Things like that and scale that as a framework that I don't think make a lot of sense either Overall I think there's some good nuggets and some good things for people to learn But it's not something I would like be excited about so So also, you know, I haven't seen the scaled agile framework work in practice And that's not that I've seen it fail. It's that I haven't seen it work in practice That said I have seen a number of other things scale Very easily to similar sizes So when I look at the teams Inside microsoft that are scaling to thousands of people They're pretty much using one of two approaches. They're either using a waterfall like plan Which works pretty well and I actually highly recommend it Adaptive or agile waterfall great technique And it works do simultaneous phases But go ahead and plan like you have for large projects with very fixed delivery dates. It works for that system Or the other one that they're doing is a very decentralized metrics based approach This is the way that yammer works for example, where they do continual analysis to understand What are the leading metrics that correlate with future revenue to what degrees of correlation? And then what are the experiments that we can do that have what impact on those leading metrics? At that point, they've now decentralized the decision of who of what should we execute on Such that each individual team can start making calls that optimize for the global whole That breaks the whole planning problem and they don't need central planning in order to get unified results and they can scale very high So I enjoyed very much what you said about scaling agile that The framework doesn't say What not to do and this reminded me of What we used at Siemens for scaling agile and we still are using it in different organizations is the book from bass potter and craig larman Scaling lean and agile Development and this is something that was very very helpful because I think in basically Every organization is different and every agile or lean adoption is different And we need to have to look at different Things that will work in this context and not work in another context and we have to know Under which circumstances we should try something and Not try something else because it would not work there And there this book was of tremendous help for us for the organization with several hundred developers also all over the world to Get to a good own agile framework that is maybe it's in some Things it's similar to the scaled agile framework But I think out of the box an agile framework will not work anywhere So I once heard someone really authoritative in the agile space say that You know, they looked at the scaled agile framework They looked at dad, which is the disciplined agile delivery and they said You know, this is exactly what we were trying to avoid 14 years ago Right, so that's someone who wrote the manifesto Making a public statement saying this is exactly what we were trying to avoid But somehow we don't seem to be learning from our mistakes That was well summarized. Thank you We had a gentleman over there and then we'll come here. Sorry guys Well, you already have yes, go ahead. Yeah Um, can I add a quick comment on the last one just We have used in our work with quality system and agile We used action research, which is not known as an agile method But something which was around since 1920 as a research method within organization is a collaborative method I just recommend you to look into that when you want to change in the organization to look into action research I think it can be very helpful. It's an agile method, but it was before agile Now to my question I have worked with I have pioneered agile in my organization and It worked very well for outsourcing from Scandinavia to India as such But one area where we have struggled has been with the product owner role we use scrum and xp But the product owner role has been a challenging one either the customer have Provided one person and it's mostly non-it companies for our customers They provided one person who've been a product owner, but they have not fully understood that role They have not wanted to participate on a regular basis Therefore we tried with having a proxy in India Who has worked together with that customer product owner and do it sometimes it worked sometimes it has not worked I'd like to have some feedback about how to work with this critical part of To get the agile to work together with customers in particular End uses and customers who are not mature and understand the agile methodology itself So I'm another bias in that I'm a product manager by trade Spent the majority of my career doing that and I think that the scrum product owner role Because it doesn't talk about the discipline of product management fully and take some pieces of it Uh It's open for sort of an asymmetry of people Not committing to doing what you need, right? I'm a smee I'm some person over the side and I want some product built But I'm going to give you a little sprinkling of my within my head as opposed to actually sitting down and going Okay, I'm building a thing a product So I think there's a little bit of How the the role is defined in the community Lending itself towards bad behavior But what I think the ultimate answer to what you're saying is that we need to evangelize that we're actually out in the world Creating more products. What do they be internal? What do they be external? We're creating a lot of products in the world We need more people doing product management and agile project product management And I think when people Have it as at least a role, maybe not a full job But when that's clear and the the norms of it are clear and what what the responsibilities are in a larger way than the product owner I think You'll you can get better engagement because it's It's core to the the responsibility of building a thing that the product becomes real to people So I think it's hard to get a smee that In a case where you get a sprinkling of their time to engage in general I think that's a kind of a flawed thing that we have So I look at this from a systems thinking perspective Um, I find that a useful way to analyze the various options. Um Fundamentally the thing that seems to determine the effectiveness of the Of the product design or the product direction will call it Is the length of the feedback loops and the number of delays And the amount of information that makes it through each hop from brain to brain loses Like about 90 of the information And every hop Causes delays and then a hop that's international Yeah Wow Yeah, so an international hop is going to cause even more delays And or more information loss depending on how it's executed So the teams that I've seen that have the best Direction with the customer have They don't even have product owner roles They have developers that are able to directly analyze the data of direct user behavior and experiment on those users directly In order to gain very good clear information. There's one level down where you have a product owner role That's doing that experimentation. There's one level down below that where you have a product owner role Who's making guesses based on their opinions and their experience? Which is now lagged information on the basis of a direct customer and then you can go further to You know distribute that or have a customer role proxy, but just from a peer systems thinking perspective It's all about how could I eliminate more hops and how could I eliminate more delays go all the way towards zero? Can I just comment on that? The problem has sometimes been that the customer is not able to explicitly communicate what they want That one I've called as a proxy has often been the one who has facilitated that Discussion that sometimes worked very well, but it's not always work. That's why I asked I've actually never met a customer who can clearly communicate what they want No, neither have I but I meant There are those who are better than those problems If they think that they know exactly what they want then you you pretty much can shut down your project on day one We don't need that yeah Proxy cannot be the conduit for successful You know product owner or a manager Proxy may not be the right word Because oftentimes when you when you send it off like this in a proxy manner, there is also he is proxy and passive also He is not empowered to take any decisions So how can he communicate to the teams? So for every decision he has to make he still has to connect back to the product owner and he's not available And he himself doesn't know So that's where the challenge is So I think that part of the so I think we all agree on But this is a problem And I think that I might have some of your same background coming from Scandinavian context myself. So how to address this problem And especially in the Scandinavian context. So I work in Norway with teams in Sri Lanka And We have some two features of Norwegian organizations that are Helping us the first feature is that People are usually open to informal decisions. So you can have things kind of happening without a formal decision being made The second one is that people like people Even though there can seem kind of cold people like to have this feeling of Of a nice conversation of like Inclusiveness is an important value. I think so what we've found to be very useful is to create a Workshop in a remote setting where we get the developers in the room and we get the product owner and the SMEs Whoever is there in the room and we sometimes have to trick them a little bit in Why we're pulling them in here? So we're saying, you know, we want you to we want to introduce a team to you We want to we want to have sort of a round table thing. These are things that are very accepted in a Nordic culture, I think and then we make that into a first situation where we Make the developers and the SMEs and the product owner Talk together and from there we're saying we like this we Suggest you do it more And and then we're and then we've got started anyway So I hope that is applicable in your setting as well Cool I have a question You need to add you could come up someone. I have a question on paid programming Yeah, go ahead. So I have a question on paid programming that is a basic question. So Can you just hang on? He needs to complete something. Please. Yeah. Hello Uh, just two lines to share is uh, we had this kind of issue and uh, you know the and we try to Ask for more involvement from real customer and so one of the reason came is I don't have time and other thing was uh Like I told you clearly what I need From their perspective and they say once it is done, then I can I can be in the feedback loop So literally our team build a fake Feature and just like a UI stuff and say, oh, it's done and the guy like a proxy Say thought it's done And then we went to the customer and had the feedback actually So that's something like, uh, you know, I think he also added like you have to come up with something That's a brilliant idea Yeah We do that with real users all the time Do you think The customer need to Need to have the agile mindset for an organization to adopt scrum That's the customer need to have an agile mindset to adopt scrum Maybe an easier question is if a customer does not have an agile mindset, what's the best thing for them to do? So, well, all right, so my challenge with the question I guess is that for for for my current role for my context The customer is an end consumer out there. That's you know, browsing around the web or has a phone in their hand Um, and I really don't care if they have an agile mindset or not. It makes it makes absolutely no difference to us Um And and some do and some don't right? So I could actually the the the question is a little difficult for for me I'm imagining that it comes from In a consultative way And in that, um Having having also done that kind of work It is far more challenging if the customer doesn't have an agile mindset You know oftentimes oftentimes we were doing transformations So they were asking us to to help them Help them get there You know, but we also just found that for a customer that didn't really have that mindset, you know a really high communication Setting expectations up front. Here's how the project's going to work. Here's how we're going to do things Here's how it's going to be funded. Yes, every you know every week or every two weeks You're going to have an opportunity to say all right. That's it. I'm cutting the funding off the projects over or we're going to keep going You know, there were some benefits that that would attract them to the idea of an agile approach And as the project progressed the relationship got better and better So there were some that when we started they didn't have really an agile mindset by the time it was done They didn't want to do it any other way So I I think You know, you don't even have to use the word agile In most cases, I would assume that customers want Value delivered at a low cost and customers want to have a good sense of progress Forget agile Almost all customers would want maximum value at the lowest cost And would want to have a good sense of progress being made So if you could have a conversation around those And expect that you know, we want your feedback so that we can deliver quickly The transformation would happen automatically So The the discussion around agile itself need not really happen. So long as customers have this broad principle in mind I jump in here real quick. I think there are different kinds of customers You know for some customers The reason to go for agile is basically I want to Or not to go the reason for some people to outsource their development Is basically they want to push the risk of their head to someone else, right? Is is that this is risky? We don't quite understand it. We don't want to do it Let's push it off to someone else and now it's their risk. So let's try and get a contract in place Let's try and get a date and time and cost in place. So the risk is off our head, right in that kind of a context How you educate the customer about agile or whether agile is even important for them is Is something to ponder upon right because Fundamentally what they're trying to do is Push the risk off their head and if you understand that and if you can explain to them that Or rather demonstrate to them that you know this technique of whatever call it blah that you're doing Can actually reduce risk then they would appreciate what you're doing much more, right? So it doesn't you don't have to go big gungo Let's do a five-day agile workshop because we're going to educate you or baptize you to to be agilist from now Right, you just you just say well what you're trying to do is push risk off your head We understand that what you're trying to do is high risk So we will try and work with you to reduce the risk, right? So that's one approach to take in some cases people want to do agile because they have heard A bunch of stuff from other people and they're trying to push you to do agile But they don't have the mindset, right and there it's already a failing game because there is a lot of Misconceptions to be clear and that that's another situation where you know, you do need to help Deal with the misconceptions first before you kind of get into the whole agile mindset with them Ironically in terms of customer the word So I build products for companies. So we do enterprise software essentially But we we build products for people wanting to do agile But It's surprising how many people want old school roadmaps So they'll tell you, uh, what are you going to build for a year? I don't know Because I'm agile about it. So I want your feedback because maybe I'll build what you want next But people like no give us the roadmap for a year or two before we buy Or you'll see people saying I don't want those releases. We're like, hey, we release all the time You know four or five times a year now we have sass. We release like, you know, we just put stuff out there We don't want that. That's destabilizing and it's like, well, wait a second. Don't you guys do agile? Yeah, so it's like, well, do you give your customer a two-year roadmap? No Then so in our case, it actually is a fair amount of friction if someone wants to buy an agile tool But they're not quite agile. It's strange So something I just want to add here is a lot of time context when here that's where context matters More than the customer customers are serving a market and certain markets are more dynamic than say other markets Or certain domains are more dynamic than the other, uh, you know domains and that dynamism in that domain sometimes, you know, drives the agility Or, you know, the requirements of the customer So and and certain domains are becoming much more dynamic than what they used to be in the past Right, uh, and that's exactly where, you know, most of the voices that you hear from customer comes So, uh, you know, you we need somewhere the delivery computer I mean the software delivery to be aligned with the dynamism of the market and that's what exactly customer is looking at Really? So a lot of the comments here have been about, uh, it doesn't matter so much whether it's the agile mindset and that it's it's much more about The outcomes and be able to get to agreement on on what you mutually share is the values that agile will help you deliver on Totally agree One of the values that your customers that almost all customers are bringing in That will not necessarily align with you is Uh, sort of what noresh was talking about not only do they want to move risk onto you They want to move executive function onto you as a customer. I want to not have to think or worry about things It's one of the biggest values that Anyone gets from buying something is I can just buy it and then not think about it Um, so that can directly get into the conflict even someone who has an agile mindset Yeah, I I want to for the things that I want to worry about tweak it all the time for the rest I want to install it once and have it work and have magic show up every couple of days and never have any Magic go away. Um, and it's up to you to make that happen um, so that's where uh getting back to some of the previous comments about customers customers won't tell you what they need Um, and they don't they actively want to not worry about this. So look for other ways to get the data user experience designers and design thinking and all sorts of techniques have been developed for Finding the data based off user behavior and the like those can supplement a more rare conversation with the customer All right, I need to call this off because people have been telling me time out time out. So We have dinner that's ready for you guys And I hope that we will continue this qna as we go through the dinner and we will Will be around here to go on. There's one question over there I I will I will take that question along with side with all the speakers and we'll have dinner and we'll take that question Yeah, this is regarding a pair programming. This is a basic question. I'm a verification engineer So we are planning to adopt this continuous delivery and pair programming for the next release onwards So right now our ratio of the developers and the testers is four is to one. So we are happy with that and What the testing has to be done within the release so by one tester So if you're going for the pair programming and pair programming and continuous delivery And this ratio is not I mean not practically doesn't work four developers and one tester Right. So what is the preference of what is the preferable ratio besides for the play programming and continuous delivery? I mean the automated everything within the release. So all this considering that so what is the preference you developers and Testers ratio People who want to leave can certainly leave. We will finish this question and then we will leave later But those who want to leave can go ahead It is all about the context The right proper ratio of devs to testers depends a lot on what the testers have to do because of what the devs are doing Um, so teams that have good strong two star agility the devs are no longer writing bugs And when they're no longer writing bugs then uh testers have a very different role Um, and I've seen a lot of companies that work very successfully with a 51 to 1 dev to test ratio That tester is one of the most important people in the company And they're doing a completely different extremely high value role They're finding all the holes in devs thinking and unknown unknowns But earlier on it should be 1 to 1 dev to test ratio because there's a different job that test is is doing So really depends on your context One of the things that I think we've gotten in a trap of With xp is the word test is overloaded So what is a test is that a unit test can be something to preserve developer intention It could be something to test the boundary condition on a small way And so because it's so overloaded everybody thinks they can test And in some ways everyone can and that's great. That's how you bring full quality ownership to a team But also the discipline of uh testing is really important. And so The ratio is again highly context dependent you want to make sure you have enough testing expertise In what you're doing so that the test you're writing the automations in you're doing is is solid And I I can't tell you what that is without knowing your your actual individual setup But I would caution you to really think about testing of the discipline And the the best functional or you know testers in the world Other people who really think about this are amazing. I don't know if you've ever seen a talk by Michael Bolton He I mean he blows my mind. I'm like wow you thought to do that thing Why did you think to test that? And so if you don't have that In your team you need to get it All right. Thank you. Thank you guys for entertaining us Dinner is served outside. It's uh, you don't need any coupons or anything You can just go have dinner and hang around we'll be around here for at least another few hours All right. Thank you guys