 Good afternoon. My name's Sean from a company called IHS. We're a global company doing data and analytics. I'm actually from Canada, although we do have an office here in Bangalore. And what I want to talk about today is agile architecture. More specifically, what is the potential role of an architect in agile? Do they have a role? What does that look like? And this is an experience report. This was an actual story that we went through in a small part of our organization that I was involved with. And so I'm not going to pretend anything in here is the right way to do things or how you should do things. This is our story of discovery and what we learned in our own process. And maybe there's something in there to take away. Quick question, who was at my talk about building a sustainable organization a few days ago? If not, that's fine. I'm not offended because I know it's the people. Is there anybody here who is a handful? All right, I'm just curious because I might make a couple of references, so I just wanted to know how much background I need to have. This presentation actually really isn't mine or doesn't belong to me. It really belongs to my friend and colleague, Chris. He's kind of the guy I'll be referencing a lot. Unfortunately, he wasn't able to be here, but he will be giving this presentation at the Agile Conference in Washington, D.C. this summer. So if you're gonna tweet it all, you can include his Twitter handle and wake him up at four in the morning back home. So I was actually really inspired by Dave's talk this morning. And then there was a question afterwards about, well, there's a lot of terms here or bouncers, I believe they were called. And maybe it wasn't immediately applicable, how do we apply this? And I thought it was a really great talk, but I started thinking, well, how can I apply it? You know, and one of the things I talked about a couple of days ago was leading by example and modeling the behaviors we want in our teams. We want teams and we want people who can accept risk, take a bit of risk, not afraid to fail early. And I thought, okay, well, how can I personally do that? So what I did is I said, okay, well, I thought Dave's talk was really interesting. Is there any way I can apply it? I know barely nothing more than that about the Kinect and Framework than what I saw in the talk this morning, but I frantically worked over lunch to figure out, well, okay, what concepts from there might be applicable to our own experiences? There's some way I can tie it together. So I'm taking a bit of a risk here. We'll see how it goes and I'm sure I'll be wrong and I'll learn about it. But I hope by being able to take that leap of faith myself, I can encourage everybody that it's okay to maybe take a bit of risk once in a while. Without further ado. This is myself, an agile coach with IHS, an internal agile coach, I work with our teams. I've got 13 years as an officer in the Canadian Army. That's me in Afghanistan. I've also worked as a programmer before. I've also worked as a product manager, essentially on the software that ran that unmanned drone. And now to set the stage for our story, I was working with a handful of teams, about five teams in Calgary, Alberta, Canada. Each team had about five to nine people. Every team was doing a mix of scrum, can ban, scrumb ban. I really didn't care as long as they were, you know, understood the principles and were developing a system that worked for them. We developed a thick client application, a desktop application for Windows. The product was actually quite mature. There was code in there that had been around for 20 years or more, like a million lines of code. It was very profitable and successful in the marketplace. Pretty much everybody worked on one product, but they were ranged in feature teams and I had been involved in helping transform towards more agility. So they'd come from a waterfall background and then maybe had been quite successful moving into agile methodologies in the first year and they were quite comfortable with that. They were in feature teams. Most of these features were relatively independent so the teams were operating very independently and it's in C++ if anyone cares. And then something happened. We said, okay, there was this initiative that had been on the back burner for a while which was we want to take what was essentially a single user application. Our clients were large energy clients would buy it. Their engineers would use it to analyze, do their work, but all their work was done individually and we said, oh, there's a big market opportunity to actually have them collaborate. Our customers and users can actually collaborate and share and exchange data so we want to have this server backend. So this was going to be a massive, massive, massive project to potentially re-architect millions of lines of code in order to support this. Now we have all these issues of concurrency, blocking, there was all these things that we had never really worried about before that we were never going to have to worry about. So this was the place where we start the story and so coming from an Agile background, I was always very purist when I came to Agile. I look at the manifesto and say the best architectures, requirements and designs emerge from self-organizing teams and I really believe this and I really did believe that it was anti-architect. If I talk to a company and they say, oh yeah, we're very Agile and then I hear that they've got an architect role, my initial cynical reaction would be, well they probably had this role before and they didn't know exactly what to do with it so they just kind of keep it around and they don't exactly know what to do with the role and it's probably unnecessary now if they're doing things properly. So I was kind of cynical about the idea of an architect and Agile in an Agile environment. And to tell the truth, we didn't actually have one in this part of the company. We didn't have anybody who was an architect so I didn't really have that problem. It's not like we had to try and actually find a role for someone. The reverse happened, we had a problem and then we found a need for a role and I'll explain that for a second in a second. So we started going, starting on this project and perhaps rightly or wrongly, what we did is we did a little bit of upfront work and thought into it that had been going on for a little time and then we said, okay, ready to go. We're gonna just put everybody on it. So you put all the teams, never really worked together before in such a way on this project at the same time, working on the same codes, restructuring things. There's gonna be a huge amount of dependencies and we kind of thought, nah, it'll be chaos at first but the self-organized and things will straighten out and there certainly was chaos and but very quickly into it, we were getting the questions, you know, when I'm coding, when should I be following an existing pattern or when should I be developing a pattern for others to use or when should I just do my own thing? I'm really confused about this. I really don't know what the boundaries are and the number of questions that were coming in from so many angles about this was causing a lot of issues. They didn't know where to begin. So we did come to the conclusion that we still need some consistency. By the very basic level, what language are we gonna use? Okay, well, it's pretty obvious we're gonna stick with C++. Then what architectural approach? Now we're gonna have to kind of look at making some significant changes to the structure. What do we use? Domain driven design, some kind of service-oriented architecture to use both. Is there something else entirely? And then how does that roll into things like, you know, where do we define various layers or components and the interfaces between them and what the rules are? And so there needed to be some boundaries so that people could start moving forward in the right direction. But what was kind of the appropriate level for these boundaries? And kind of typically they're very structural when you talk about architecture. You know, you have some coding standards, that's defined some layers, that's defined some components, responsibilities between them and their interfaces. And we said, okay, we probably need something of this so that people know where to start. So we didn't have an architect role, but we needed to put some thought into this to make sure that everybody was on the same page. We didn't want to do a whole bunch of upfront designs. So we said, well, who's kind of got the most knowledge, experience with the code, put the most across the product, depth of product knowledge, maybe put the most thought into what the challenges we're gonna have moving forward. And then that's my friend Chris. I stole this off Facebook without telling him so he has no idea, his picture's up here today. And we said, okay, we need to solve this problem. We don't know how to do it. So Chris helped figure it out. And I mean, it sounds kind of un-agile-ish, but the reality is there's a lot of supporting players in place there. But we never actually made them, like you never gave them an architect role. So even to this day, I don't think we really know what his job title is outside of what the HR system is, nor did we really care. This was about solving problems and our culture anyways that job descriptions weren't that big of a deal to us. We were more focused around solving the problems, so I still couldn't, I don't think architect is quite the right word for him, but nobody else knows what to call it. So anyway, so that'll be my symbol for Chris moving forward. But we knew we wanted to avoid this, right? This is often what I've seen architects turn into, and I've witnessed this in other companies where it's kind of ivory tower architecture, right? I am the technical expert who has a very broad knowledge base and I am the one and only person who can come up with designs and solutions and then I write them out in very elaborate documents and UML diagrams and I pass them down and it's other people's jobs just to implement things, as I say, they should implement things. And then we create big batch sizes and we don't empower people and so we knew we wanted to avoid this. This is not what we wanted at the end of the day. But we had to start somewhere and what did we come up with at first? Well we came up with a very simple architecture design to get people kind of going. So this is roughly what it looked like. There was maybe like five or six blocks on a page that had names and a handful of rules that said what each block was supposed to do and how they were allowed to talk to the other blocks. And then we printed these out on big posters and stuck them up around the office and maybe had a couple sessions with the developers to explain what it meant and that was our initial tangent and we were hoping that this was gonna be sufficient, minimally sufficient to get started and actually it for the most part was. It was pretty good and it became down to the design level. So in each one of these blocks when the developers start talking about classes and the interaction between classes and how these classes should be designed and arranged which largely is something that we expect the developers to be able to do. The challenge is is a couple years ago it came from a waterfall background where everybody was very siloed. They weren't used to talking to other people let alone other teams entirely. And so we didn't have a lot of that ability to have those good discussions and so what we decided to do where we thought we'd give a try was we need just enough information. Does these user stories roll up through the backlog? We need just the right amount of information to maybe solve some of the challenges so the teams can take that and run with it when they get to a user story or a feature. And so this was kind of what we tried. This is our first attempt. So the architect is a scout. So they were gonna go ahead and provide reconnaissance on the things coming up. So they would talk to the teams, understand what they were working on now, some challenges they were having, looking at the backlog and seeing what things were coming up and then they would go ahead and spend their time prototyping some things and actually writing code coming up with one or implementing one or two or three different designs and discovering those unknown unknowns that maybe were going to bite us in the future and do some evaluation of the pros and cons and then take that information and filter and pass it back to the teams so that when the teams came time to do those user stories that they had some of those uncertainties removed for them and this actually worked not too badly. It was a reasonable start. We came to this epiphany that it was actually essential for the architect to be connected to the code and Chris would tell you like if I wasn't actually spending time coding, the code was so fluid, it was changing so fast, so many people were working on it. There'd be absolutely no way I could do this and totally abstract block diagrams. I really needed to have that thorough knowledge of what was going on and I needed to be coding to be able to do that which is often not what our impression of what our architects do. So to try and pull this back to Dave's talk this morning, I believe this would be an example of disintermediation. Let's remove the layers in between the actual data and then the decisions about what we're going to do with it. You can correct me if I'm wrong about that but that's my first shot at making that connection. So this is okay but we ran into a few issues. Inevitably information got miscommunicated. It happens. People don't quite have the same conceptual integrity as someone else. We don't necessarily have the conceptual integrity on a design as we're talking about it. Information will get truncated. You'll think you're passing on one team. We'll talk to another team but they won't necessarily give them the whole story or realize that they need all that information. The information would change over time as we discovered more and more about our own code and what we needed to do. Now the problem is this kept on coming back to being the architect's fault. It's kept on coming back to Chris whenever something like this happened and the teams would get all frustrated and say, oh, this is your fault. You're supposed to be solving these problems for us. You should be working harder and harder and he's getting quite demoralized at this point in time because he was trying to do all he could to pass information back but these are going to happen. They're inevitable so we couldn't be perfect about it but so how do we get around this that it was becoming one person's problem because we knew that's not where we wanted this to be. We wanted everybody to have ownership of the problems and work through them. And this manifested itself in a lot of team friction. I'm not sure if anybody has experience in seeing multiple teams work on the same thing and seeing this team friction and one source of friction we noticed was different design values. Now we're putting them in a situation where they need to collaboratively work on a complex software design and they're working together to do object-oriented design and come up with these designs and then implement them and inevitably what would happen was someone would develop, one team would develop for their user story a module that was supposed to be reusable. They were gonna use it but then it should be reusable in the future by another team and another team would go to reuse it and they would say, well, this sucks. And they'd say, well, it sucks because it doesn't follow single responsibility and we love single responsibility. That's what we should be doing. And the other team would say, the team would wrote it that say, well, too many classes make things too complex. So you quickly discovered that we weren't even on the same page with what our fundamental engineering design values were. We also discovered there was all this miscommunication that happens. A team would try and talk to another team to explain what their needs were and then they would build it and then it wouldn't quite be what they want. And then again, this would end up being the architect's fault for not properly making sure all that communication happened. I mean, you do the five whys behind this. We came to this epiphany that we were missing design skills in the developers. What the developers were actually asking for if you were to solve the immediate problem. The immediate problem was they were saying, give us more and more information and give us more and more direction and be more and more precise in the designs you want us to build because we don't want to be wrong. And if something does go wrong, we want to be able to point our finger at someone and we were missing this collaborative environment and because we didn't actually know how to collaborate. The reason we didn't know how to collaborate is because, well, it was quite obvious when we looked at it and said, well, they came from an environment where they had their own code they were responsible for. They didn't need to work with other people or evaluate designs. So they didn't even have the vocabulary or, I mean, these are very smart people, by the way. I mean, they've been working and they've been programmers for a long period of time, but this was something they never had to do before. They never had to talk to people about their designs, evaluate different designs, do it objectively, take into account engineering trade-offs and come to conclusions that, the best engineering conclusions. What you would see is it was often this emotional situation where, well, that code sucks. And so very quickly, communication would become very hostile. So we saw this was actually the real problem we needed to solve. And so, go try and tie this back to Dave's again. I mean, this was this distribute cognition and sense-making within the network. So we had this network of developers and we needed to be able to embed the ability so that we could distribute this sense-making amongst them as opposed to trying to solve what they were asking for was just tell us what to do. That wasn't going to help. We needed to not do an organizational hack and solve the short-term solution, but we needed to invest and find a long-term solution which actually meant investing in people and their development. And something else happened at the same time. We discovered that we were woefully wrong. In fact, in the initial architecture, we had come up with, or that initial tangent, it ended up being completely, we're not completely wrong, a major problem with it we discovered. But in fact, this was kind of expected, right? We had prepared ourselves for the possibility that we may have done something wrong. And that's why that diagram at the beginning was as simple as it was. We said, okay, we think this is where we need to go. Let's start working on it and then we'll see where we run into a problem rather than spend months and months and months trying to be perfectly right. So actually, this worked really well. We discovered the problem early. However, that didn't stop the programmers having a different perception of this. To them, this looked like, to them this looked like, ah, again, the architect was wrong and therefore now it's going to cause us a lot more work. Because we hadn't included them as much as we could have or should have in the actual discussion. So they weren't aware of the reasons and the rationales and the thinking that went into place to that original idea. They didn't have any kind of investment in the original concept. And so it was just someone else's own that and then the consequences were trickling down and causing them more frustration and work. All right, so at the same time that this happened, I started circulating this video from David Marquette about, it's called Greatness and he's an ex-US Navy submarine captain, nuclear submarine captain. I think he does a really good job of explaining how can we empower others. How, I won't go into the details, I'll encourage you to watch it if you haven't. Who's seen this video, by the way? Hands up. Okay, great video, please watch it. David Marquette, it's called Greatness and an Innoversity did all the animation stuff for it. Chris was really inspired by this. He said, you know, this is really, really interesting because it's related to the problem I'm having right now which is teams are asking me for all the details. They want me to give them the answers but what we really need to do is really give them the tools and the power to find a lot of these answers for themselves because otherwise I'm always going to be the bottleneck and that's not really in the principles of what we're trying to achieve with agile. So it's another epiphany we came to. Chris realized he needed to, this was really a leadership problem. And by leadership, he had no people technically working for him. There's nobody under, he was not a manager of anybody but he needed to influence the organization and he actually needed to influence it without any authority whatsoever. So it's almost the ultimate leadership challenge. And this is not usually how we think of an architect, is it? You know, the architect is usually the person who's got all the technical skills but no leadership skills. Therefore we kind of, well, what do we do with them? We promote them and not always. There's lots of good ones out there but I have seen it where, you know, they get promoted up into the, you know, because it's just a technical role. They just need to come up with solutions and pass them down. But Chris came to the realization that, no, no, what I need to do is I need to develop people and that's a leadership task. I need to develop others. And so, how did we go about doing that? Well, in going back to Scrum and the Genesis of Scrum and you've got a team that doesn't know how to collaborate together and come up with an organizational hack to get around that and encourage them to collaborate so you have a daily stand up. So what did we do? We came up with this, our own organizational hack to try and get teams to collaborate which was this Scrum of Scrum-ish type daily design meeting. We didn't really have a good name for it. Kind of looks like Scrum of Scrums except it's 30 minutes. Anyway, so what did we do? We found, we developed this idea of the design quarterback. You know, each team would have someone who would be very fluent in object-oriented design principles but also able to communicate it well. And there were kind of the teams really driving force and they weren't gonna be a dictator. I mean, this wasn't micromanagement but they were gonna be the technical competency that was going to drive development and growth on the team in that area. And so what did we do? We take one from each team and they were selected kind of in an un-ajah way actually. Normally we say self-organization. Anybody and everybody can be invited. Well, there's always been that opportunity for anybody and everybody to get engaged in this discussion, but it wasn't happening. And we wanted to encourage people to think more and more about design and architecture and thinking about things and communicating and collaborating and doing it in a kind of professional collaborative way. So we took the people who had the best potential exhibit at the best characteristics. Didn't matter what their seniority was. Some were scrum masters, some were managers, some were leaders, some were the most junior guy on the team. Didn't matter, it was totally a merit-based decision. And we said, and this actually caused a bit of an uproar, whenever he said, well, I wanna be involved with that and I wanna be involved with that. And it was interesting, we talked to someone else in the company and he said, what people actually want to attend a meeting? He's so used to the idea that what people hate going to meetings. We said, actually, people wanted to go to this meeting, but we said, anybody can go. However, you have to have demonstrated this fairly high level of competency in being able to communicate design. And then the idea was that the architect was actually this kind of scrum master of the session. So every day they'd have 30 minute discussions and they would bring their problems that their teams were having and challenges they saw and the idea was, okay, let's figure out how to work through these problems. Let's actually understand the best way without getting emotional about it, but how to solve some really, really, really tough design architectural problems. And the idea was that the architect was not actually giving the answers, he was asking the right questions. He was encouraging people to think with the right thought patterns. And then these guys would go back to their team and then they would essentially repeat the same pattern. They would go back and have built their own skills up, they would go back and have the same kind of discussions with their team and then ultimately it was bringing everybody's skillsets up together. And this was really tough because Chris said it this way, you know, when programming reached this concept of tell, don't ask, you tell an object what you want, you don't ask it for information and then operate on it. You let it operate on the information. Well, you discovered when you're in this leadership role, it's the opposite, you ask, don't tell. Don't tell them what to do. What you do is you ask questions to help them think through the thought process and get confident in that ability. So it was a very different shift in his own way of thinking. And so what did we learn from all of this is we really came out of that understanding that the architect was this kind of technical servant leader, that technical skills were essential. It changed the role from this idea that their job was to come up with designs and then pass them down to the teams and the teams mindlessly implemented them. Because what we ran into was that when you do that, certain assumptions go into this, you know, into the initial architecture. You know, you put all this thought into it but there's all these assumptions built in and when you simply pass that down without thinking, people start implementing it blindly, not recognizing when one of those assumptions might have been wrong. Because they really have the information. As they're coding it, they see the code, right? They know, but they would miss that opportunity like wait a second, we designed our architecture this way because we thought this but now I'm finding something that might be contradictory. Maybe we need to close that feedback cycle and go back and re-evaluate. But because they hadn't been involved before, they didn't recognize when that happened. They just kind of blindly go ahead. So now that there was this more collaborative approach with that risk largely went away. We had closed the feedback loop. And so kind of where we're at right now is this idea that the architect, you know, is a coach of design skills. He facilitates design discussions. He helps improve people's ability to then facilitate their own design discussions. This architectural scout thing is still kind of, he does, you know, very much integrated in what Agile is and really trying to leave the thought on how Agile fits together. And we'll spend some time going ahead and being this like, what's the new and next great big things coming down the line? Just before I was about to leave on this trip, I'd been away for some time and then I talked to Chris and he said, Sean, you've got a problem. While you're away, something happened. We discovered that we don't think we need this meeting anymore. And we thought, oh my goodness, what was this presentation like, meaningless now? And while he said, well, we discovered we don't really need it anymore because people were starting to, you know, the teams were coming to this discussion and they'd already solved the problems. They'd already solved it on their own. In fact, they were now talking to each other on an ad hoc basis. As they discovered something comes up, they would literally walk across the hall and have the discussion and do it in the right way. They'd be thinking about the right things. They'd be looking at the trade-offs. They'd be doing it objectively and they'd come to something that they could commit to and move forward. So the value of the meeting itself as this organizational hack to force people together had largely diminished. So in many cases, this is a great success, right? We've now invested in creating habits and behaviors and we don't no longer need this artificial ceremony. So it was a transient state. You know, as Dave was talking about this morning, we were in this transient state and that's okay and we evolved and so did the role of the architect in this case. Now what does Chris do now? Well, you know, we've got this job security not necessarily role security. Well, he's now has time to do bigger and better things and more exciting. He's still with us. He's just, of course he's still with us, but he's doing what I think we should all be doing which is trying to over the course of several years make ourselves irrelevant, right? We work to develop others and so that they can do our jobs hopefully better than we can and that increases the capacity of the company. Now I'm free to look at even other places, other ways we can innovate. And so that actually was an interesting development at the end. So in conclusion, I guess what could we learn from this? Any kind of architecture role or any kind of role that's looking at the bigger picture has to be interconnected to the code. You can't separate that and expect good results. They have to be able to go in there and code and do it on a fairly frequent basis. The architect is this technical servant leader. They're coaching and developing others and to do that they have to be developed in leadership skills. And for Chris this was a big kind of wake up call for him when he realized that because that wasn't something that, maybe it was his natural strength. This was something that he had to deliberately and consciously work at. But in doing so he's probably more eloquent about it than anyone else. Because it was unnatural, he had to work harder at it and put more thought into it and now he's quite eloquent when it comes to talking about leadership. And a solution's not gonna work indefinitely. It's going to evolve and change over time and you have to be prepared for that. What you might need today is not necessarily what's gonna work two or three months from now. Maybe four months from now we'll find that Chris needs to kind of get involved again. We'll see what the future holds. So apparently if you put a hashtag and no one in front of something it becomes really popular. I'm gonna try something. I like to experiment. I'm gonna try no questions. Not that I don't love questions but I'm not gonna pretend I have answers. I might have some ideas. But I'd like to encourage people to have conversations. I'm more interested in conversations. So I mean whether there's anything here that was interesting. I mean and you wanna discuss with your colleagues at your own tables or you do have a question and you might be interested in what other people's questions are. Come up and we can take the last few minutes to discuss at the front here in just a conversation. Or if you'd like to go in and talk with other colleagues out in the hall that's perfectly fine or email me afterwards. So I'm trying to maximize the amount of actual communication that goes on rather than have me try to do a really bad job of answering someone's question in a large group setting. So we'll experiment and let me know if you like this for that or not. Anyways thank you very much and yeah we'll see you at the rest of the conference. Thank you.