 team. Are you able to hear? You guys are able to hear? I will repeat. My name is Abhinav. I work for Walmart Labs based out of Bangalore and I work for the program management team there. With me I have Bhavin Kamane who works with us. Today we are going to talk about our journey and the learnings from that journey in perspective of these three important areas program which can be agile only and agile program. So at Walmart we realize that the regular processes that are available on the shelf don't necessarily fit into our kind of working or the existing structure. So understand this better we have about 130 agile teams who work on a few dozen programs with different business priorities. They have different release timelines but they also have dependencies within each other. So we came up with this custom process that you may not necessarily find anywhere else and the way we came up with this process is that we kept on re-prospecting our own process and found out what is best. Make sure that every business stakeholder, every person who is involved in our project has the same vision of the goals. So what we do is we get the whole team together as along with all the business stakeholders, people who are interested in this project or programs which will be dependent on that project and we talk about the key business goals with the team and have a common vision and understanding of what we want to do. As a result of that we come up with this top business ethics and kind of sign off. When we say sign off it's not frozen, it's something that will revisit later but to start with that that's what we have. I'll talk about the exception process in detail in my latest slide. The second step of our process is a sign off app. They also decide what will be the key contents of that release. Again this can be revisited and corrected based on business planning. Something unique that we have added is called iteration zero. What we realized while we were executing this project is that there is work to be done for work. We want to set up an environment. We want to get some sign offs. Some teams that have been operating already need a four hour iteration zero. There are teams that need a big long period to set up their environments, their bills to get ready. Otherwise the first iteration just goes in that. We allow our teams to do this iteration zero. Then we go to iteration planning which is regular where teams decide these are the key effects that we have to target. These are the stories that will find detail breakdown of the stories and define their tasks and execute. Then we start iterating. Within an iteration we have the regular cycle with the daily standards. They relook at what's happening and eventually they do what they want. I'll talk about this in a little more detail later. There is a regression. You must be wondering what is regression to do there. What we found out is that we have a large e-business setup that's already existing. Any change to that system has to go through a change control right now. We want to move away from change control. We want to build our processes in such a way that there is no change control and we are able to identify it. But right now we have to make sure that any change in the e-business that we make goes through a regression phase. That will be extremely helpful. So what we did when we said service, we have a team that started to build a service that is completely out of order. The team plans its iteration, usually a complete iteration and they keep building that software. They can deploy it in an independent environment, not necessarily dependent on anybody else and people can use that service. And what we could have is a set of services which have similar technical goals or similar business charter. They form a group of services. Each of these services' teams build their software independently. Now that's fine. We need an alignment with these. We want autonomy, but we also want an alignment. So what we do is we define before these teams start executing. And we build a chart plan for stories. So a mini workshop on that where let's pick up some existing stories from a backlog and let's start decomposing that. So what happens is that's where people will start getting a feel of how do we implement this into our real life. So in the end basically what we realized is for any scale you are looking at macro impact. But those macro impact can come only with micro level changes. Those changes need to happen at an individual level, at a team level. And together that's what ends up creating a macro level impact. So again, from a model perspective, the vision was basically we wanted team to deliver early, deliver fast, on-demand delivery. So have that capability where they can get into this whole continuous delivery model. So while doing that basically they should continuously introspect, retrospect what is happening, what is learning. So that learning has to be there within the team in terms of they are learning from what they are doing essentially. And whatever learning they have, any takeaways, any experimentation, any output that needs to come as part of their continuous improvement. And this is a sort of a cycle of I don't know is there an end to it or not. I don't think there is an end to it. It's all about just making it more and more continuous. And just keep empowering the team so that they take decisions rather than people from the top pushing the process down. That's about it. Just to summarize. So what we realized that we have to sooner or later move from a governance, agile governance based model where we push changes top down to enablement kind of model where we enable people to retrospect their own processes. The key takeaway is the process has to be retrospected itself and the changes have to be made based on your own situation, own condition. Every business case is different. And for us right now this is the optimum solution. So process retrospection is our key learning. And team enablement or governance model is the other takeaway. Any question from the, yes? So let me answer that. So we make every effort to co-locate teams as Babin also said that we are trying. So our model is this now that development engineers, QA people, product managers and project owners are at least this set is at one place. Given where we are, the kind of businesses that we are involved in, say for Bangalore office, a lot of business is in US market. So a lot of business will continue to be there and it makes it more effective and meaningful to have business there. But the whole effort right now is to make sure that people are in one floor, in one place. That has given us enormous benefit. We had, one of the examples is that we had these support teams, say for network engineering, database, security teams. They were different teams, right? Who used to work in different geographies or different offices. Now as a result of this change, they all did this retrospection and realized that they need to create these smaller agile teams which has a presentiality from each of the domains and they get co-located. They still work in these horizontal functions where they get to share their knowledge. But from a physical point of view, they are one team, they have one backlog and they can pick up anything. It is possible to create some sort of a mechanism in which we put these teams in not more than two locations. So it may be practically impossible for big businesses like ours, for that matter to co-locate everybody. But essentially we said, okay, we will use max two locations. So as I said, we allow a level of autonomy, but we make sure that the integration in the lifecycle is also defined on, and we have this continuously. The use cases that I was talking about, the vertical ones, the ones that keep the team-binded all the time. And we have also made sure that there is, you know, the way people are hired and employed, they do that kind of management. Measuring team sentiment. Can you elaborate about it? Because as agile teams, we tend to glorify the delivery rating or the product owner rating on the demo. But I thought that was an interesting, what does a team feel about itself? What kind of metrics do you use? So it was really simple. We had smileys, basically. It's like emoticons, really. And three simple or two simple emoticons, you know, basically you have sort of a red, yellow, green kind of a sentiment, right, essentially. Basically, we just expect the team, you know, the whole team to basically come together and come back with one consensus, right, essentially. And that happens with every iteration, actually. So what you want to check is, you know, at the end of iteration, it's not about velocity, but it's also about how the team is feeling. I mean, velocity could be great, but the team is basically coming back and saying, just to share an exact experience, personal experience. So I project managed one particular team whose velocity chart was, like, spiky, right. Their emotion was always great. They always said, we're feeling good. We were very worried about the delivery from that team, and that was the team that delivered on the same day as I expected. It's a soft thing. We don't know, you know, how to, kind of, how does the team arrive at that? But it has a value. Sorry, we'll take the position. A task. And they are more looking at the task, and later you transform it into a story. So I do see that only a few things were added, which is your user story was there, and your requirements, and this can go with it. So how do you really say that still the team was getting a lot of input out of it? So what we are doing is, if you look at the board, what is moving is story, actually. So let's go back to, that's right. So if you look at this board, really, each of this sheet is nothing but a user story. It represents user story. So we are not plotting in terms of task, in terms of, you know, as an individual where my task stands. It is more around, you know, checking the flow of the story. And each, so basically what this kind of says is that if a story is a waiting period, it means, you know, UI is yet to take up a story. Right? So these are all stories, and none of the post-its were really task-persuited. Just to add to that. So I'm sorry, it's not very visible from where you are. So what the team found out, and of course our coaches helped us realize that any user story flows through some states. And this team realized that for almost all stories, they have these four states where a piece of back-end code has to be written, a service has to be written. There's a UI that has to be built. Two, that UI and service has to be integrated. Three, it has to go through quality certification. Right? So instead of, they did two things. One, they made the story short so that they could flow faster. Now they could find out in what state is this particular story. One of the findings, for example, was that most of our queue, which was getting bigger and bigger every day, was the UI queue. And we realized that most people are efficient in developing back-end code, and they're not efficient in writing UI code. And it was very evident. The team felt that, and they took the corrective action. So we're making the story flow instead of a task move. It doesn't, from a business point of view, it doesn't mean much. This campaign's result has to be moved to the other side. Dependencies, how they were done? Dependencies, that's all. Yeah. So one of the ways we look at dependencies is we have this inception, where we, inception phase that we defined. The first state that we have, where we identify the first set of dependencies. Right? That's one. Two is the use cases that flow across programs. And we, every team publishes a plan on when and how they're going to integrate. Right? That's two. Three is that we have a concept of users, of a stories. So we use software tools like Zera and Confluence, which help us, especially Zera with Greenhopper, helps us to build a story in one backlog, have a mapping story in some other backlog, and build a relationship between these two. Or a set of stories. And when I build my dashboard as a project manager, the first thing I see is, what is the state of my dependent stories? And similarly, the other project also knows what are the in stories. Just one second. One second. Is that complete? And then we'll get to that. So in the inception process, you talked about architecture, right? I would imagine at that time, architecture would be at a very high level. Right. How is your process to break it down into more solution architecture at least so that some teams can start working with it? So the question is, during inception process, the architecture that we talk about is pretty high level. How do we break the architecture into smaller components and detailed architecture so that teams can start working on that? So when we do architecture, it also goes through an architecture review board. That's where they look at the external interfaces and the whole ecosystem. Why does it fit in? So once that is approved, it is the system architects and the team who then start working on the detailed architecture and close it. They are very closely part of these. So that doesn't really mean that you have one architect who's full-time into just one team unless the team holds a very specific position. But most of the architects are shared between one or two teams. That's kind of an architect for a program, basically. And they are continuously there as part of the stand-ups or planning or the way stories are elaborated. All of those exercises happens with them. Really. You had a question. Does that answer your question? Yeah. So still I didn't question your question. Cost-planning. Sorry, that was not within my jurisdiction to do a cost-planning. From an effort and time-based? And yes, we did that. We plan a project on how many people will be required. But then we keep on changing that plan as and when needed. But in terms of dollars, no, I don't do that. That is, based on you having an initial thing there that said you had to do iteration zero and stuff to help you set up, which suggests to me that you're having this to ramp up the ramp down teams. So are the teams, do they tend to be stable? Is that what your demand profile looks like? Or does it kind of go up and down a little more spikey? So we have had examples where teams were reprovisioned because this project scaled down and this moved up. Typically, we try to move the whole team as much as possible, but we don't have a policy as well. But it happens more like a real-time business. So we have programs where they have agile teams which don't have a specific charter. They pick up any project and start executing. That's one model. That's one model. The other one is that we have a program and they're a dedicated set of engineers for that program. Yes. Oh, yeah, very good. I mean, so when we say team, the team definition for us is all of these people. QA, architects, UI, developers, you know, the product owner or business analyst, all of these people are sort of there. That's what we call it as a co-location, not just a slice of it. So to add to that, there is an important responsibility that our QA teams have is to help the product owner build the acceptance criteria at a finer level. So we believe that they are the best judges of what should be the fine, detailed, granular acceptance criteria. The product owner has knowledge at a certain level and then they help them build and align the acceptance criteria. So they help build the product backlog as well. So again, for the automation, the idea was, you know, of course this is not like a brand new project that is just coming up, right? There's a huge legacy around it, right? So the whole idea was anything that, you know, the team is building, they want to automate those things, right, essentially. So the first focus is to automate already for things that are in queue, right? And if there is a bandwidth or there is a need, then you can have a separate user storage just for automation. QA plans are essential, right? There are levels, right? So within the constraints of a new program, most of the testing for that program at the yellow lines that I showed in the E2 end-to-end release is within the scope of this testing team. But we have a lot of legacy, you know. We have an existing e-business site or number of sites that are already there. So we are periodically making changes to the system. We are making sure that the existing system does not break. We have no big-bang approach to replacing an existing site with a new one. So we're doing the incremental changes and that is where the regression testing is required. So when we add to the existing one, we have to make sure that. Okay, go ahead. We'll take these questions. Just to give you the development or they have handled the support of the current products that they are working on. And if it is the latter, usually that screws up the estimates for your students, right? You don't know when a bug is going to come, how much you will spend on it. So we have faced that problem. Our initial plan was that you own what you build forever. So that model still exists, but we have built levels of support between the dev team. So when something goes into production, we have levels of support, L3, L2 support, and then eventually it reads the dev team. We have a concept of dev ops, takes up the tickets, site ops that takes the tickets. It depends on from which environment the issues are coming. So if it is a production environment, it goes through regular support channels. Eventually it can reach the same team that developed the code. You can just suddenly, you know, meet us actually. I can talk a lot about it. Thank you a lot.