 All right, my name is Naresh. I'm going to be talking about this topic called organizational resilience. Just curious to know how many people here are in a position where they can completely restructure an organization. Quick, show of hands. That's awesome. This talk is more geared towards giving people ideas about what kind of thought process they could use when they're looking at restructuring organizations. It's a case study. It's based on real-world experience from a couple of companies, but I'm going to try and stick to one specific case study where we're going to go into a little bit more details. But I also want to set a little bit of a context, a little bit of theoretical context behind the underpinning of how you think about organizational resilience. And that's kind of where this talk will come from. Just to quickly start and get everyone on the same page, I want you to name a few companies that you think are impacting you every single day. Google, Amazon, Facebook, Scrum Alliance. Interestingly, all technology companies. Hindustan, Lever, absolutely. All those fabulous companies. The question is, will the success of these companies last forever? Can't say. We can be pretty sure it won't last forever. How long is the real question? And I think this whole talk about resiliency is really trying to figure out this answer. What can we do to maximize our chances of lasting and being impactful as long as possible? Would you agree? That's kind of the premise of people talking about resiliency, organizational resiliency. This is why people care about it. I want to set a little bit of a context and talk about some thought process around when we think about resiliency, a thought process that comes from the adaptive change cycle thought process. How many people are familiar with the adaptive change cycle thinking, this model? I've presented this a couple of times before. This is just a starting one-on-one thing. If you look at nature, there is a lot of inspiration we can draw from nature. If you look at how forests are built, typically they start off with some basic flora and flora trying to figure out how to survive in the weather conditions, in the soil conditions, in the atmospheric conditions around them. And out of vast majority of combinations, a few, very few of them actually survive and grow into what we call as these dense forests. So they go from an exploration phase into what we call as a conserving phase, where they try and maximize and extract resources from the nature. What kind of resources are we talking about? This could be rainfall, getting water. This could be sucking out the nutrients from the soil. And they grow this way, and they become really big, dense, and heavyweight. And at some point, they become vulnerable to things like forest fires or some kind of a disease. And they almost very immediately, catastrophically fail and fall over. And when they fail and fall over, they release whatever energy, whatever things they had sucked out, and release it back into the nature, which leads into what we call as the fourth phase, which is the reorganization where now new combinations of species try and emerge out of whatever energy that was released back in. And then they go back into this exploration phase, and then we kind of continue this infinity cycle. Does that sound familiar? Let's take an example if it doesn't sound familiar from the automobile industry. If you look at the automobile industry, a bunch of tinkerers were trying to figure out how to move beyond horses, how to do something cool. And it was really bunch of tinkerers trying to explore, trying to figure out what can be done. A few of them figured out combustion engine, a few of them figured out how to get wheels, a few of them figured out how to do things which worked in this context of automobile. Very soon, some of these explorers got into this conservation mode, and they grew into this mass production where you would produce thousands and thousands of cars every single day. What happened in the conservation phase is that a lot of these companies very quickly fell out of sync with the natural demands, the climate conditions, what people's consumption patterns are. You might have heard this popular saying from Henry Ford, which is not true, but I'll still say it. Henry Ford saying that you can have any color car as far as it's black. And the idea was basically mass production, no personalization, any of that stuff. Which led to, after maybe a lot of conservation period, it led to the downfall of some of these gigantic companies, which led them back into this reorganization and now people are exploring with all different alternative ways of personal transits and so forth. So we go through this cycle every few years in different industries, and this is something that we all are kind of seen and are familiar with. So when we look at this as a backdrop of this, we ask ourselves what does it take to build a resilient organization, something that can survive this adaptive change cycle that is very much built into the nature of how things are. Yeah? Yeah? I'm trying to say this is the background, right? There is this natural force that goes through this adaptive cycle and one needs to figure out how they can stay ahead of this and not get basically destroyed because of this. And that's where I think resiliency will make a lot more sense and some of the things that we can do in resiliency. Before we go into that, it's the most important part of the talk, right? So why am I talking about this topic? So my name is Nareesh. I am an adventurer sports freak. I live in Mumbai. Don't act in Bollywood yet someday. I was a part of a company called Directive. It's recently sold one of their businesses for $900 million, one of the largest acquisitions. I was part of Industrial Logic, which used to build e-learning funded by some of the big companies. I started another startup which failed. It was a kid's educational startup where we were trying to build games to help kids learn mental mathematics. I've been responsible for putting together a huge variety of international conferences across things. Convengen is a platform that I started building as something to scratch my own personal itch. And it's grown now about 100 conferences used this worldwide. I was part of Hight Messenger, which is India's fastest unicorn. Happened to be India's fastest unicorn. And my latest venture is Accentio, a consulting company where we help businesses figure out how they can engineer their DNA to really deal with the whole digital transformation that's going on. So with that enough of me, let's go back to the talk. Let's take a minute and write down your thoughts around how would you define organizational resilience? What does organizational resilience mean to you? Just take a minute, put your thoughts down. Any volunteer who want to share? Yeah? Warren Buffet says, how big is your boat? Moat? How big their moat is? That's an interesting way to look at organizational resilience. You had a point? Cool. So her definition or her way to summarize what resiliency means is an organization that is constantly staying proactively staying ahead of the curve or at least at there so that they are staying current with whatever is happening. Would you put companies that are disrupting other companies as resilient? But may or may not. We'll have one more quick definition. Actually, we'll go there. She's been putting her hand for a while. To remain profitable, ways to remain profitable. So if an organization stays, continues to be profitable and grows, you would consider them to be resilient. OK? That's the way to do it. It's cool. I just wanted to get some feedback. I have a definition which is not my definition, but something that I like. So I'm going to just play a quick video from an organization called BSI, which talks about what is their definition of resiliency. Is the audio on in this? It is. Today's interconnected and volatile business world, the ability of an organization to anticipate, prepare for, respond and adapt to change is more important than ever if they're to survive and prosper. A resilient organization is one that not merely survives over the long term, but flourishes passing the test of time. That's it. That's their definition. I think it kind of captures the essence of what people want when they think of a resilient organization. There could be a lot more, but this is simple enough as a working definition. So with that, we will talk about what will make an organization resilient, right? What are some of the important components that make an organization resilient? Again, I'll just continue the video to hear what their perspective is. We realize there are core components at the heart of organizational resilience. It's about delivering the right products and implementing the right processes with the right people. Thinking organization-wide about operations, supply chain and information resilience. It's about becoming a more adaptive, agile and robust organization and acquiring the habits of excellence and continual improvement. Peace about continual improvement is something that I think most people in the agile community will resonate with, right? That's one of something which is very core to the whole agile philosophy about continuous improvement. And so I kind of like some of the components that these guys are highlighting that makes an organization resilient. Why am I playing these videos? This is why we reinvent the wheel when someone's already done the work, right? But it's not all videos. I'm gonna get into this. There is an interesting thing that a professor, David, has come up with where he talks about how, when we think about organizational resilience, how it's evolved over a period of time, and so that's gonna be the next video of how organizational resilience has evolved over a period of time. The report identifies that the thinking on organizational resilience has evolved over five distinct phases. The first phase was very much about stopping bad things happening. It was about preventative control. It's about having systems and processes in place. That then moved on to an agenda which was very much about having people who were alert, attentive, careful, mindful of what they were doing in order to avoid trap and mitigate problems. The next phase of organizational resilience thinking was very much about a progressive agenda. And there are two aspects to that. The first aspect is around performance optimization, which is about doing what the organization does. So it's existing technologies, it's existing markets, existing processes, but doing that much better. That's opposed to the fourth phase of resilience thinking, which was very much about the organization doing something radically different. So to be the disruption in its marketplace, that requires adaptive innovation. The fifth phase, which we're calling paradoxical thinking, is about blending the former phases into a more holistic and blended fit for purpose approach for the particular context that that organization faces. All right. So those are the five phases through which, according to Professor David, are thinking over the period of time as evolved in terms of organizational resilience. And I'm gonna touch upon this in the case study that I'm gonna talk about how we went through these five phases and what it means in the context of building highly scalable products. But just to summarize for the benefit of everyone, preventive control, mindful action, performance optimization, adaptive innovation, and paradoxical thinking. These are the kind of five distinct phases that we go through. But what do I do with this? How does it help? What do I do knowing these phases? How does it help? It's actually, the next part of the video is very insightful where he talks about how, you know, we actually came up with these five. What are the tensions? What are the drivers? And once we look at that, I think we will get right into the talk. Within those five perspectives on organizational resilience, we find two core drivers and two core approaches that organizations can adopt. The first driver is a defensive agenda, which is about stopping bad things happening as opposed to a progressive agenda, which is about making things happen in the organization. In terms of approach, what we find is some organizations drive for resilience by trying to strive for consistency. That's opposed to a flexible agenda, which is having people that have a variety of different ideas, a variety of different beliefs and outlooks, and a variety of different behaviors and practices, which enables them to be much more agile. Now, that creates four core quadrants. In the bottom left, we have the preventative control. That's about defensive agenda by being very consistent. So this is about putting defenses in depth. It's about stopping bad things happening. This is opposed to the bottom right-hand quadrant, which is about mindful action, and this is having people who notice the problems in their environment. They're able to raise those concerns and those concerns are listened to, and they're empowered to act, to take action, to stop the problem cascading and escalating through the system. Now, in the progressive agenda, this is about making things happen. The top left-hand box is about performance optimization. This is about the organization doing what it does now, but doing it better to drive competitive advantage. Now, this box is opposed to the top right-hand box, which is very much about adaptive innovation. This is about being the disruption in the marketplace and no singular approach to organizational resilience actually achieves performance in the longer term. It's actually a blend of these different approaches. The challenge for organizations is that there are tensions between the different quadrants. For example, when an organization feels a threat, it tends to retreat into the bottom left-hand box, which is about preventative control, so it puts more systems and processes in place. It controls resources and it controls its people even more, which drives out the flexibility and the agility that it needs to perform well in a dynamic and changing environment. Right? Does that make sense? So just to recap quickly, there is the progressive agenda and there is the defensive, which is basically defensive is about stopping bad things from happening. It's the companies which are risk-averse, that kind of outlook. There is the progressive, which is making things happen more opportunistic nature of companies. Then you have the flexibility, which is consistency kind of tension, which is consistency from my point of view is basically organizations which are more process-driven, more output-centric. And flexibility is more around people-driven, which is allowing the flexibility based on people and more outcome-centric. This is not saying that one is bad versus the other, just acknowledging that there is the spectrum, which goes from consistency to flexibility. So if you put these four together, you get what is called as the tension quadrant, which is essentially these four quadrants, which maps back through the phases that we talked about, the five phases that we talked about. So the first one is the preventive control, the first phase of organizational resiliency. Then we get into performance optimization, which is the second phase, mindful action and adaptive control. And then finally you take those four and you try and figure out how can you bring paradoxical thinking to achieve maximum organizational resiliency while covering all these four quadrants. And depending on your context, you may be heavier on one side of the quadrant versus the other, yeah? So that's kind of in a very small gist, if you were to think about organizational resiliency, this is how you would try to map it out. You would look at various circumstances and you would see on which quadrant you want to maximize your investment in some sense. When people talk about this and something that I had written in my abstract is basically can we apply now, plan to check, which is something that we have done quite effectively, you know, the plan to check act kind of a PDCA, which is what people refer to, to deal with achieving this kind of an organizational resiliency in different contexts. And that's where I think again, Professor David has some interesting insights and he says that we need to come up with a new methodology which is beyond the PDCA approach that we have been used to. We also propose a new methodology which is about how you achieve organizational resilience. And this builds on the two aspects of approach within the tensions model, the one on consistency and the one on flexibility. Now, existing approaches of plan, do, check and act, which many organizations know and have used fits really well in terms of the consistency agenda. What we're proposing is the foresight methodology to drive the flexibility agenda. The foresight methodology has four aspects to it. The first aspect is foresight. Foresight is about anticipating the threats and opportunities that your organization will face in the future. Insight is about understanding the present conditions of the organization, what the situation is and how people are acting. Oversights about measuring and monitoring the system and having the checks and balances in place. And hindsight is about learning and learning from experience. The foresight model is particularly useful for organizations that are operating in more volatile, uncertain, complex and ambiguous environments. Organizations need to be both consistent and flexible. That requires a blending of the plan, do, check, act methodology and the foresight methodology. This really requires a shift of thinking from either or to both and, and that requires paradoxical thinking. Does the foresight methodology seem familiar to people? We may not use those exact same terms, but it sounds familiar. It's things that we've been trying to do without explicitly calling those terms. Would you agree? Some of us are nodding their head. Some of them can relate to it. This is just to summarize the four parts of the foresight methodology, which is what I believe a lot of times in product development companies have been doing because there is a lot of volatility in the context that they're operating on. And so you come up with, again, the lean startup guys have talked about this in a slightly different terminology, but they have something similar going on in terms of that cycle, the build, measure, learn cycle. So now let me tie all of this back. So far, this was really theoretical, trying to set the context. Let me try and tie this back into the case study that we have here for today. So far, are you guys with me? It makes sense so far what we have covered? Just the theoretical underpinning for what I'm about to talk. So how many people are familiar with this style of an organizational structure? Everybody should be, all right? Is this a resilient organization just by looking at the structure? May or may not be, okay? Let's assume that this organization is not resilient. What do you think are the problems with this structure that makes it not resilient? What kind of things can go bad? Hang on, one person at a time. So interfacing, there's no connection between them, so there's no interfacing between them, okay? This is more of trying to show the organizational structure, but yes, there should be some interconnections between them for them to be able to operate. But fair point, hierarchical and rigid, that's interesting that you make that connection. I mean, that does not necessarily need to be hierarchical or rigid, but usually is in our experience, okay? So that's a fair point. The customer connection is missing. Who's responsible for customer interfacing? Okay, that's a good point. They're working in silos. Each of these are working in silos. And what does silos do? Like what's the problem with silos? Each team's maybe operating in silos. They may not have a cohesive strategy. And so when a sudden change happens outside, if there's an external change, they may not be well equipped to respond to that quickly. Making sense? All right, anything else? We have a gentleman there. So the four things that we talked in the previous, the new methodology that we talked about, that seems much more harder to do in this structure. Why is that? Silos, because of the silos, okay? Okay, so similar, again, silos seems like a problem. We need all of these to be able to offer something to the customers, but in this structure, it doesn't look like it is easy to do this. And hence, when there is some kind of an external change or external event that happens, a competitor releases something. In our case, Facebook released something and we were screwed overnight, basically because of some of these challenges. You had a point, yeah. That's a great point. If there are new opportunities out there, the current structure doesn't naturally lend itself to be able to leverage some of those things, and which is where you may fall out of date, or you may not be experimenting enough, and hence, you may be disrupted quickly, right? So these are some great points I would like to continue, but you guys are with me, right? So far, this is some of the challenges with this. So this is not resilient. So what do we do from a structure point of view? This is kind of what we ended up with the structure in our team. Sorry for my bad choice of colors. It's very hard to see, but what you can see is we ended up taking each of those silos, grouping them into what we call as value teams, and those value teams had a dotted interaction which is a central, like a virtual team, what we call as the leadership circle, and that had representative from executive leadership, but also from all the teams. And this is where this may look a little familiar to people from the cheshire kind of a background where it is more of the circles, and then there is a larger circle which has representatives from this, and there's a two-way arrow, which means there is a two-way sync between these. But the important thing to notice is the top one is just a virtual team which is essentially representatives from different areas coming together. What does this kind of help us achieve, right? So let me just define what a value team means in our context, what does it contain, right? So value team basically focuses on end-to-end ownership of the customer experience, of the delivery of that experience, and discovery of that experience, right? So it is, in our case, we had a value team which was a games value team, so they would build games on the platform that we were building, and they would take end-to-end responsibility of building that games, right? We had another value team which would focus on the end-to-end experience when it comes to messaging on the platform, or reading, consuming some kind of a content on the platform, or making payments on the platform. So you could have multiple different kinds of end-to-end experience, each one being served out of a value team, and they would take the whole ownership of the entire end-to-end experience. These were self-contained, no shared resources. When I mean resources, I'm not talking of people, I'm talking about how these operated in terms of complete independence, complete autonomy to run as they wanted to. A lot of times we should, we would say that these are startups within the startup, you know, that the idea was to create that kind of an ecosystem of startups within a startup, and I would say they were more anti-fragile, or at least that was the thought process that basically even if one part went down, the rest of them would continue to operate and take advantage of it, rather than everything getting disrupted when things changed. So when you look at this, what we try to do is we try to define certain kind of OKRs, objectives and key results, at the organization level, and we try to drive those OKRs across the entire teams, all the value teams, and use that as a way to come up with the governance. So what are OKRs? Are people familiar with OKRs? I would assume, yes. Most people are. Unfortunately, I won't have time to go into the details of what OKRs are, but one of the key questions you would ask is at an organization level, for example, what would kill us tomorrow, right? And you would say if we don't do this, if we don't have this, if we don't have this, if we don't have this, then we're basically dead in waters. So typically that was the driver for us to come up with three OKRs at our org level, which was around engagement. So if our users are not engaged on a platform, we are dead in waters. If we are losing our customers, if we are losing our users, then we are dead in waters, right? So retention was another kind of OKR for us, not an OKR, but an objective for us. And then active social network, which is basically how actively people are engaged on the platform, how many other people they are bringing to the platform was another kind of a measure. Now, when I talk about these three, these are fairly generic. We have applied these in other kinds of context, and we've seen that these kind of high-level OKRs seem to work really well in their context as well. I would call these as pseudo-measures. These are pseudo-measures because they actually, you know, you can look at profitability and you can say that my profitability is going to go low if any one of these three is going to go down, right? So you can actually keep them as placeholder measures, which can then help you derive a lot of other, you know, specific measures without having to get into those specific measures just working at the abstraction of these three levels. So what we came up with in this, this was after six months of iteration, is with these three OKRs, which is engagement, retention, and active social network, we basically, so those were the desired outcomes that we had, we converted them into objectives, and we started with the org level. A lot of people do ground up, which is they started the team level OKRs and then they kind of build it up. We did the opposite, which is we started at the org level and we then started bubbling it down to the team level. We went all the way to the team level, but not the individual levels. Again, a lot of companies use OKR at individual level. So each person in the team would have a set of OKRs. We thought that was an overkill in our context. We kind of only went down till the team level. And essentially this gave us a way to measure how each team is operating, how each team is performing or contributing. And this felt more like apples to apples comparison rather than other measures that typically are used. So that's just a little bit of a context. This helped us with the very simple governance for us. A, we started with democratizing the data. So everyone had access to all the data. How many users, what's happening with them, which feature is contributing to what? All of those insights were converted into dashboards easily available to every single person. Every month, we basically had what we call is a evidence-based show and tell. So you have to say, here are 20 hypotheses that I ran last month. And out of those 20 hypotheses, here are the results. These experiments have matured and they've got into production. These are experiments that we killed. And that was kind of the thought process with monthly show and tell. Waterly, there would be a review. We would look at how each team is performing. And if there are high-performing teams based on those OKRs, they would get more funding. They would get more things. If they were not performing, then those teams would get dismantled and they would have to go back to the pool of people that would then get sucked up into other teams that were performing well. This is basically our governance model in one slide, right? This is kind of how we structured our things, ourselves. Starting with restructuring our teams to value teams, then moving from there to defining org-level OKRs, bubbling them down till the team level. Each team could say out of these three OKRs, I will contribute to these two OKRs by 5% by 10%. And on this OKR, I will keep it as is without making it worse, right? Those are the kind of commitments each team would make. And then quarterly, we would review how each teams are performing and decide whether they need more funding or they need to be dismantled. So far with me, why was this done? This was done for us to basically focus on these three things that we were trying to achieve. The first one is a user-first thinking. It was very important to create this whole culture of user-first thinking rather than developers and testers or engineers saying, oh, this will be a cool feature. We want to build this feature. Or marketing people saying, oh, this sounds like a great idea. We should go ahead and build this. This was more of user-first thinking, which meant we would run an experiment. We would have to have a clear hypothesis. What are you trying to do? How does this improve any of the three OKRs that we have? And if you can show with a statistically significant number that you can actually impact it, then it gets rolled out to the rest of the user base. If you can't, then it basically gets killed, right? So that's the whole user-first thinking kind of an approach, aligned autonomy, which meant that every value team was completely autonomous. They could make decisions for themselves, but they had to be aligned in the sense that they all had to operate off the same platform. They are not building completely radically different things of different platforms. So that alignment was very important. And the last thing was basically de-risking ourselves. The previous years, we had ended up building a lot of stuff, pushing it out there and realizing that it didn't meet the expectation of what we wanted to achieve. So how do we de-risk ourselves? How do we know well in advance that what we are attempting is not going to be a flop? We're not going to put something out there and it's going to end up being a product debt for us. So these were kind of the three things, if you will, a culture that we were trying to create which had these three components or these three pillars. So now if I go back and map through the things that we talked about, which is preventive control, for example, the first stage of the resiliency, there we put in a lot of extreme programming practices and continuous delivery practices to make sure that the organization was able to cope up and was not making, had things standardized across these teams, had the ability to quickly respond as things changed, if someone makes a change, you know what's going on, you get a feedback. So some of these practices from extreme programming and continuous delivery really helped us get that preventive control resiliency level, if you will. From there, we moved to the aligned autonomy that I was talking about where each team, each value team had the total autonomy. So they could sense, they could respond in their context and they had the autonomy to kind of operate independently. This allowed individual teams to be much more responsive than trying to control this centrally with one team trying to decide everything for everybody else. So this nature of just breaking them down into individual startups, if you will, gave us a lot more mindful action, you know, form of organizational resiliency. Then when we come into performance optimization, there was a lot of things that we were doing in terms of safe rail experimentation by bringing in data analysts, by bringing in customer insights people into each team. So each value team comprised of these folks and they would run a lot of safe rail experiments and they would optimize things, they would optimize for value, they would optimize for things and they would try and maximize what investment we've already made, how do we make sure that we are getting the best out of it? So that was about performance optimization using some of the safe rail experimentation thought process. When we come to adaptive innovation, this is where are people familiar with dual track agile, dual track implementation, where you are doing discovery and delivery kind of simultaneously together. So that's referred to as dual track. So on every value team, basically, the same set of people would come up with a hypothesis, they would go run experiments, do user research, figure out what is working, what is not working with low-fidelity prototypes, with high-fidelity prototypes, do a lot of experimentation, then figure out what works for them and then kind of the same set of individuals within the team would go and kind of build what we used to call the skateboard version of it, which is the very minimalistic version of it, put it out there to 1% of the users, see what is the, how our hypothesis is actually measuring against that. Once that seems to make sense, then it would get rolled out. In a lot of cases, we would just basically kill the feature much before it gets scaled out. So that's the adaptive innovation where working very closely with your end consumer, trying to understand the difference between how a 17-year-old person in Kolkata would be very different from a 15-year-old person in Mumbai, trying to build a product that caters to all these different sets of user segments, meant that as a team, we had to work much more closely with these guys rather than relying on some product guy figuring all these things out for you. So this is again about that adaptive innovation only happens when you are very close to the problem, when you are kind of dog-fooding and working very closely with people. And finally, the whole structuring of the teams, I think, into this structure allowed them to take these four and bring in what I believe paradoxical thinking into the organization. And that led to the organization being a lot more resilient in my opinion. This is just a quick summary. So last year, we had shipped 107 features before we made these changes. And the year we made these changes, we ended up shipping only eight features into production. So you might think, like, okay, this organization is not becoming resilient, it's just becoming stupid, right? And I would make that kind of conclusion as well if I didn't have the next slide of data, right? Which is, this is an email from the founders and at the end of the year to everybody saying, I know a few of you have doubts about how much progress we've made this year. The feeling exists because we've only shipped eight features. So we don't have what I call the illusion of progress, but I wanna highlight some things that we have achieved. 30-day retention was 2X up, right? So we improved the 30-day retention by two times. The engagement buckets were up by 2X as well. And most importantly, our monthly active user base was, you know, that went up by 50%. So at least if you were to summarize this, shipping more features does not necessarily mean you're gonna get better traction. Shipping less, but shipping the ones that you really tested it out and you know works is what really mattered. And how do you get to those things that really matter is where I think some of this thinking behind the organizational resiliency and how you operate much closer to the problem kind of helped us kind of achieve that. So that's kind of quickly what I had. I know I've raced through this, but hopefully that kind of gives you a perspective of how we use some of the early theoretical things that I talked about blended that into the structure and how we came up with some governance, very lightweight governance to structure our organization to bring in a lot more resiliency into the organization. Will this make us resilient? It's hard to say yet, but it feels like we are a lot more resilient because we're competing with some of the really big gigantic companies out there and seem to be kind of continuing in this race, still like surviving pretty nicely and kind of growing the user base very organically. So it feels like at this stage we seem to be doing well. Of course, time will have to say how we end up where we end up with, all right? So do we have time for questions? Two minutes, perfect. Yeah, right there. The governance structure that you said. Yeah. Okay, so we have the value based teams and at a higher level, an organizational layer just to govern. How do you enable collaboration between these teams? Because what I understood here is that in case a team is not performing, then they will probably loop back into an organizational pool from where they will get sucked in. So how do you enable collaboration between these teams versus an in-fight stating that I need more because the resources are always limited, finite. So just a thought wanted to me. So we, and I'm gonna say something really controversial. We did not encourage these teams to collaborate a lot. We wanted these teams to operate like they were competing with each other. So there was not much of collaboration between the teams. It happened informally, but there was no formal structure put in place. What, the only way that teams would learn what other teams are doing is A, they had access to everyone's data because data was democratized. Every month, everybody had to showcase what they have achieved. So there was knowledge sharing and learning happening in that sense. But there was no active role in trying to get them to collaborate with each other on a daily basis because we wanted to leave them to try and figure out an experiment on their own and kind of have a bit of originality of thought rather than kind of getting too much influence by each other, if that makes sense. So they were kind of pretty much left in silos, if you will, to kind of operate, but they were at least on a monthly basis, syncing up and learning from each of those experiments and what was going on, all right? Then you talked about how did we decide because resources are limited, how did you decide that? For that, what I was talking about is you would look at the OKRs, how the teams are performing on the OKRs. If the teams are performing well, then they get more funding, which meant they could get more people, they could get more access to servers, resources, et cetera, et cetera. If the teams was not performing well and if consistently you see over a quarter, which is three months, if they have not improved any of the OKRs, then they would basically get dismantled. And then? They started maybe. The AWS CTO. Okay, I think I got your questions. Your question is basically, is there data around how many experiments we ran? How many experiments were basically killed before they got out? I can pull up the data. I think it was 700 some odd experiments. I don't remember the exact number, but 700 odd experiments were done and killed at various stages. Some were killed when you had the first contact with the user that this is stupid. Some were killed at a much later stage. But it's a good thing that we did 700 experiments and we killed them. And that's again, to me, a feel of how resilient we were. Cool, thanks. I think you had one question. So one was building games, another one was building payments, things like that. So they were fairly independent, but they all worked off the same infrastructure. So for example, a user is shared across all of them. When someone's running an experiment, they can't run an experiment on the same user while someone else is running an experiment on the same user. So while they were fairly autonomous, there was a layer of dependency that everyone had, which basically we had done like AB frameworks and stuff like that which allowed each team to operate fairly independently. So our view was like we are AWS and there are multiple people operating on their AWS. So how do we enable each team to be self-service kind of a model? And there were times when each of the teams would contribute something to this fundamental layer which would allow them to do self-service and other teams to do self-service. Correct, the velocity is a terrible idea. Business value is not something that we can really measure, but what we could measure is the key results in the OKR, and so we would focus on the key results, but everyone focused on the key results. So if you're building a game, someone else is building something else, it doesn't matter what you're doing. What you're optimizing is to improve engagement, to improve retention, and to improve active social network. So no matter what feature you're building, no matter what you're trying to do, you're focusing on those three things. You wanna make sure that each of them are maximizing those three things. So if this team contributed 5% to increasing the retention while the other team didn't contribute anything, right? Similarly on those two other parameters, if you look at the teams, one team is contributing on all three parameters, for example, one team is not contributing, then you clearly know that there is a problem, right? It's not comparing how many features you shipped. Nobody cares about that. It's about how you moved the OKRs, that's what matters. All right, I'll be around if you have more questions. Thank you.