 Hello everyone and welcome. Can you hear me all right? We tried to make this as much to give this as much contrast as possible, but Can everybody see this? All right, so I think let's start So who are we? I'm Tasos Kutlas. I'm senior technical consultant with Cameron Wilding in London My name is Alissa Stringer. I work for Investors and People. It's kind of the client within Presentation, so I'm currently a product owner within that Can you hear me better now? Great Yeah, so Alissa Stringer, one of the product owners within Investors and People. So in the third entity within this work was Big Blue Door, which was Doing the the technical delivery of the project. So Your words about the Investors and People? Yeah, so essentially this talk is going to talk about how those three partners have come together On a key project to look at our digital platforms of our Investors and People So a bit of background to Investors and People in the organisation So we are the UK's leading people management standard. We offer accreditation for employers. So we're looking at Recognising and celebrating high-performance cultures within organisations Originally set up within government within 1990 and we're kind of commercially driven now and financially self-sustaining We've got a large client base of kind of around 14,000 accredited organisations Across the UK mainly and some internationally To give a bit of a context as to why this project came about so we launched a new standard So that's kind of the framework and the content at the heart of Investors and People in September of 2015 Alongside that we're looking at our digital platforms and how we deliver that Accreditation via that and also via offline methods. So this project really looked at Delivery and iteration of those platforms in order to support that new product We manage the products on the brand at a head office team very small team And that's why kind of we rely on external resources and we'll we'll talk more about that obviously Purpose of this presentation. So why we kind of worked with tassels at Cameron Wilding and why we kind of outsource our external development team And we also have a kind of a larger infrastructure of delivery teams and delivery network. So So us as Cameron Wilding we were located in London in Elephant and Castle for You know We we do primarily for things we do we help organizations Transition into an agile way of thinking and delivery of the products We help with strategy UX and design and we are technology partner as well So we are specialized in Drupal and open source solutions. So we we do that for them We can follow that link and see some of the work we do and clients we work with So just to give you a bit more Context into the situation kind of pre this project and pre kind of our changing ways of working So we are as I say our investors and people very small team We work therefore with our development partner, which is big blue door and we wanted some resource as kind of technical lead But also to kind of help us transition into an agile way of working and consistently apply that across investors and people So we have kind of a core user base of our platforms So those are individual consultants delivering investors and people to organizations Using our platforms on a day-to-day basis to help them do that And we were looking at iterative development to support those people We obviously have a very large client base and we also have platforms that those can also engage with So we're talking quite quite a large client base We had complex six systems and multiple digital products And we're looking to consistently apply some methodologies to that and do some kind of fast and ambitious work with those We have limited in-house technical expertise Hence why we work with kind of external development partners So we're looking to bring about a culture whereby we could start to share that knowledge and expertise We had a very traditional approach to project management we were kind of heavily specking up front and Kind of allowing that to then go on technical side and not being too involved in that So we really at the heart of investors and people brought about kind of a product team A very small team and looked at how we could start to translate requirements and work with the technical guys To actually deliver core functionality and be involved together as that part of that process We had a very patchy approach to testing what we found as we were releasing functionality that was very buggy And it was causing us a lot of issues and releases weren't very scheduled And we were looking to consolidate that So Tassels will talk you through kind of the key challenge. Yes So we identified some of this in three main areas First was to identify the nature of the bags and stabilize the digital platform And that was due to the fact that we had every trust in everyone Everyone involved in the project, but yet we still were having bags So there was something there to be done and we needed to identify why that was happening and how we could actually Mitigate some of those factors second was that We wanted to create something that it could then scale up to other products and be the base Yesterday base for for delivering new innovation going forwards and the third one was to transition There was a there was a a strategic Decision from from investors in people that they they would like to work more as a lean and now outside organizations So they could get more innovation forwards so They the real issue was that we had to achieve that within six months because that was the budget allocated And that was the the period we had to actually show that the way that we were suggesting to work with that would be It can be done in a way. So When I was called in to the project and I met the team I Had to assess the situation and so as we usually do with all of our clients We suggested the spring zero. So that spring zero was a discovery mechanism for me to understand all the the the different the different issues and And what not within the organization so We identified Issues in three main categories, so There were communication issues within the team and within the general organization And I'll get into these into more details in the subsequent Slides, but essentially the three three main areas was communication the project delivery and the quality of what we are getting so First communication so Sometimes we say especially in our style that it's the interactions between the people are more important than the Than the tools and the processes we have Which is true, but these need to happen within a structured way So what we understood when we got into that project is that there were too many different tools where to use were used to track those conversations that they were happening and The discussions were happening across different mediums and tunnels. So emails were Used at some point and then conversations were switching into phone calls between Specific individuals and then a slack channel was there and some other individuals having discussion on that and then There were also different trackers tracking progress of specific things and then they had comments and that Was doing was was not doing the project any justice at all because essentially The whole message was diluted between those channels and the mediums and no one actually knew what was the current version of any Conversation so some people might have a different version of events or not as updated as somebody else within Because they weren't part of that conversation And on top of that we had too many people involved at different levels And that was causing problems because then the message needed to be contained It was not contained in a way and it was across different people Another thing that we've seen about communications was that there was panic when bags were introduced and there wasn't Any any real process to actually deal with those issues So depending on the nature of the bag and who was answering the phone or was capturing that information within the the backtracking system and They had that then to take it up to themselves to put it across to the development partner And then come back to a solution and then the whole release that needed to happen was was completely sometimes was miscommunicated and and and that was causing more issues for us and then I'm sure that you've seen that in other projects as well So they were a set of internal stakeholders that they were leading discussions And they weren't allowed the team to to be autonomous and get to decisions plan the work and then Understand how they they're gonna deliver it So since especially the last piece was was one of the requirements that Came from from our development partner as well because they were frustrated by all of these too and they said you know Sometimes they weren't very clear what the priorities are and it was very difficult for them to actually do work For us in a way and and and deliver in a consistent manner. So we thought that This has to stop and we need to sort of like create a framework that would allow investors in people to To go Back into a process that can scale. So the things that we did we didn't reinvent the wheel or anything You're you're not gonna see any big Eye-opening things in there or any wisdom. We just went back to basics on what agile is behind the process on the principles and what it tries to achieve and so one of that is it tries to achieve collaboration between individuals and Allow for discussions to happen, but it does that Especially in scrum you do that very structure. So there's no external noise. So the team can be You know where you can feel safe that once that discussion has happened Then they have the time to deliver on that and then that discussion can happen again in a predictive and coherent manner. So First thing we did and we identified is that the team working on the project needs to be guarded of all the noise so We we provided the single point of contact and that's the product owner Alyssa in this case So Alyssa was responsible of getting all the communications from the internal stakeholders and the the client and then get these And see she would own the communication in a way with the rest of the team to Deliver the vision and make the team understand what the business value of what we're building is And that was about it on on their own then, you know Using in other slides then the whole responsibility of actually delivering was placed on the team And that's the the trust issue agile and tails so as I said before we had few a few mediums and channels to talk to each other We tried to to cut things down a lot So we standardized a little bit and we said that pivotal which is a tracking system Specifically for scrum So pivotal was our main point main main software to capture the requirements and get the work done We started using slack for any or all discussion. So any discussion outside of slack was for us non-existent as if it hadn't happened and For synchronous face-to-face communications. We started using Skype. So If you ever worked with a remote team, you would know that this is not a solved problem Even in 2016 we don't have a video conferencing software That's good enough to actually say, you know use that and be done with it So we had to go to other solutions like Google Hangouts Go-to meeting and depending on the situation. Sometimes Internet was not great your side. Sometimes internet was not great in our side. But yeah, we we had to To be a little inventive on that on that front Result was though that we ended up working much more smoothly and everybody felt Everybody felt that they had Some more control around the processes and the structure of what we were doing so a little bit more so and that was Essentially what we were doing with With communications. I think it's safe to note here that Once you start building up the team trust and you start building a team to work together Then those especially if you have a process communication issues become less and less because People are people and we have social beings. We work with each other We learn each other and then at some point we we we become we become friends and colleagues and That affinity allows us to actually deliver work much more closely in predictive manner together So the the second pain points Area of pain was the product delivery. So one of the biggest things I have found out discovery was that There wasn't so much involvement in I from IIP and that was directly opposite from from an agile perspective because I Mean a few years back when most of the project were waterfall. There was this kind of Of notion that you know, somebody would compile a very big document of requirements and then throw it over the fence Someone to deliver but you know in an agile project. That's not the case anymore You need to have a product owner who's from the business who holds the the value and the vision within the team to Questions and constantly communicate back to the team why there are things that we need to do and then Get all those discussions that happened within the team and move them into the business and Allow that discussion to steer stakeholders discussion as well. So That was not happening at all. So we had a few There was confusion around the Requirements we had and some of them are very high level and some of them were very low level. So At some points the The work that was landed over the our delivery partners were not Couldn't be delivered because it was very high level and sometimes it was so minute that they were leading them down to specific paths When in fact it shouldn't So we didn't have that that feedback loop from them either. So that's the last point actually says that what we wanted was Because we had an exceptional delivery partner and we wanted to capture the expertise and the experience they had on on delivering those Projects into our way of thinking and then help them shape the product as much as we save it through through the vision so Some agile elements were there others weren't but as a whole that process was not Was not sustainable so Again, you'll see here. We just went back to the basics of of scrum but We actually explained why a few things have are happening that way in Scrum and what's the agile? Principles behind of the so we stopped having Requirements there wasn't any requirement again. So there were a few Spreads it's Excel spreadsheets that they contain strands of the work that we were doing that they were they were capturing the thinking but as Alissa was our productivity had all the answers we Here and I we started using The user story format. So we identified and we understood what kind of users do we have within the platform? And then we tried to capture those requirements from that perspective and I think that's really important because that makes It gives you two advantages basically the first advantage is that it's sort of like separates pet projects or preconception of people that they have of what the project should do because once you put that from the perspective of the user then you need actually Validating and verifying in a way why you need that functionality and the second thing it does especially with the acceptance criteria is it gives you like a roadmap of what that story should be and and I'll show you how we did it in a second but the way we have acceptance criteria Was as questions They weren't statements. They were questions. So the The answer should be yes. So an acceptance criteria for a log in page for instance is Can I enter my password? Can I enter my email? Can I press a submit button? Am I logged in so for for a story to be done? Then, you know, you need it to be able to answer. Yes to all of these but in essence if you wanted to understand what the acceptance criteria is you need to actually understand what the story should do and the story then is linked back into what the value the value creation piece through the user and that's a whole It's a big discussion. We had them. I mean, you remember, I mean Initially when we started to write user stories even for a very long very small pieces took us hours But then we streamlined that process more and more and it will became much easier. So That was one piece of work. The second piece of work was to introduce sprint goals And even though that seems a very simple or small task Once you have a sprint goal, then it's really easy to allow the team to be autonomous because You don't need to be part of the decisions anymore As long as if a developer has a choice of doing a or b Then, you know, they should do whatever is closer to achieving this print goal it's as simple as that and you delegate that decision to them and The the only problem which is a problem and they they should raise their voices if any of the choices actually Delivers on that spring goal because then as a team we need to go back and understand whether that screen goal is is is Is a correct spring goal that we have or not and if there's any problem within that so I Think that was a very good idea for us and works really well We had the full team participate on daily standards and we had we started off with a few standards and internal External so we scrapped all that we said one stand up same day same time same Every day and everyone to participate have a little bit of a cat's up You know, if there's anything blocking anyone basically and move on So we strictly kept that into 15 minutes making sure that everything's okay And you know identify if any conversation needs to happen and then we moved on We started having sprint planning sessions That was not there So and then within those sprint planning sessions, we we split them into two two sort of like Sessions if you will so within the first one we had we were identifying what was important and what was the The the user stories that we will be discovering next and within the second part of the sprint planning We were identifying individual tasks within every user story that needed to happen So we sort of had a top-down and then bottom-up approach So that gave us a very good understanding while we were going into sprints of How good our estimation was and essentially allowed us to become very stable You see at the result section how stable we became in terms of velocity and In terms of having the same understanding of everyone of those user stories of those user story points And how much work they actually were We introduced baccalaureate final sessions and we introduce a spin Review sessions and we allowed the product owner to be our ambassador to the business within the spring review So it wasn't our job anymore to deliver the project but that was like a celebration and the product owner was was Was demoing and reviewing the sprint with the stakeholders and everything everyone else and to celebrate what we achieved in the previous two weeks, and it was In terms of functionality we achieved really really big issues really big strands of work and functionality going forward So just to sum up what has come allowed us to do was to collaborate On the strategy and the technical implementation We improved the tracking of of of our work and the visibility Of what everyone was doing and we we encouraged more open more collaboration from the the stakeholder team and we we had a way to monitor bugs and Capture that work within our our normal our normal work. So that identifying and fixing bugs made was made much easier and within and much more predictable so One One byproduct of working that way was that the team had complete alignment So instead of having a client and vendor relationship and anymore anymore We had one thing and we were all working about the same same same vision. So we we had complete alignment. We shared the product vision Expectations were clear roles and responsibilities were clear to everyone and we had a very consistent framework to actually put Foot-forwards and deliver that work. So just some say boost about reframing the work And I think essentially that's the implementation details perhaps of of of this particular work is That's a user story. That's a user story. That's the format we use for everything So the top there there is a paragraph that's describing the functionality from a user perspective as Alisa said the the standard is delivered through Partners those are called practitioners or delivery partners That's what you see in there and then it's the standard format that you see in in every agile book About user stories. I want to something so that the value that's gonna be essentially created but we we kept our focus on that value a lot and we I mean I Know that, you know, you can everybody can say, you know, that's the format that you need to write the user story And then you know off you go What we did was to to go back again and again and again and re-validate, you know that value I mean actually making sure that the conversations are happening is this valuable enough. Why do we want to do it and all those kinds of questions and Alisa's work was to come up with the answers and Making sure that those answers then are communicated with everybody so they can understand second piece of work and this is the extensive criteria you will notice the the question format I was saying But that it was the conversation so with what you don't see is that below this user story There was a whole stream of comments from the development guys from Alisa for myself And everybody else concerned with the project that we were discussing and we were understanding what the story We had different strategies. We perhaps in other situations. We had to split stories Or we were linking stories as in this And I think people tell is is an exceptional tool to it's the best tool you can have to actually Deliver a scrum project. It's much simpler than anything else out there And it makes it really easy to link and have those conversations And very streamlined user interface so I think I Think that allowed us to to have the conversation and not have any requirements at all But use that format to to put forwards what needed to be done so the final set of of Officials we had with it where we think quality so There was a quality Assurance process in there, but it wasn't always enforced and that it wasn't always enforced because it it depended on who picked up the phone Sometimes there was a confusion around the environments and where people should be testing things before moving to production And what the deployment process should really be there were inconsistencies on how the specification was captured around what was wrong with the system and Requirements and Strunts of work didn't have test plans or specifications attached to them. So Once we we were moving forward and releasing then we couldn't go back and do some regressions around some of the biggest things so Again, we didn't even the wheel, but we were very very persistent on some of the odds are principles The definition of done just a simple document, but it's really powerful. It means that Definition of done is a series of steps. Every story needs to go through In essence if you go through everything in the definition of done and all of your acceptance criteria is a yes Then that story it's done if not Then we either haven't captured correctly what needs to be done for that story So we need to add the new story to correctly capture the the functionality or we have a bad and That streamline the process for us very very well I'll give you an example of our definition of done document in a second But what what is important here to say is the definition of done needs to be the same for every story So it needs to contain all the steps that need to be done to to for that story To be to all the steps that the story needs to go before it's it gets approved by the product owner or whoever has the approval capabilities in the project alright, so Two things just to improve quality and making and making that development process as transparent to everyone as possible Was to have a specification written before any code is written So a developer was sitting down and he was saying you know that is the functional Specification of what this story should do and this is how I'm gonna see it So for instance for a story That is about creating a content type and then style it into a specific style It was saying that I the functional spec was I understand that these needs to happen and then I'm gonna go into Drupal I'm gonna create a content type It's gonna have that machine readable name and I'm gonna add fields to it gonna feel at the field of that name Into there that's gonna be a text field. I'm gonna add another field. It's gonna be a text area sort of like that And because we use Google Docs That was shared with everyone And within the user story in pivotal So everybody who wanted to have some visibility or wanted to all senior developers who wanted to understand What the solution of a of a more junior developer is they had a forum to actually discuss those things and That discussion actually helped us a lot to identify things or implementation details that they wouldn't scale early on before even any line of code was written and That helped us a lot to actually produce software what was much more scalable and Had had gone through that review process Even has started in the review process very early on so the second thing that That the software developer was actually then doing was writing the test plan that comes back to the discussion We had yesterday about quality. So I think it's important I know I think I think it's imperative to have to for a for a software development at this time in age to Write the test plan of their stories. They need to sit down and think how someone else comes back and Uses the functionality and that is the only way that can happen because if we take for instance the login story They need to understand how somebody would go through the steps of actually going through that that functionality So if you are to write a test plan about Testing the login and that's before even any code is written, right? You need to go back say, okay, there's gonna be a text box There's gonna be a box of some kind. It's gonna have two fields On the first one it needs to have email and then that's when it kicks in and say, you know Do I need to have a method there to identify if something that's an on email just to make sure that people sort of like get I don't say the resources on the server for instance or not and that that that's what makes the discussion Starts happening. Even even if it's not between people even if it's on the developer's head They start understanding what actually the work really is And that's early on again. We need to spend as much as little as possible to to understand the work and then allow them to go into development. So Second was this plan and then it makes it easy for everybody else to actually jump in and test and you don't need to have Dedicated testing team to do that I mean within the same people within the same team within the same screen and within the same story You can have people who are external to that code because they are not working directly on it and have a test run Go through it and make a make sure then that at least function that works And what I consider this is the first step to getting into automated tests as well Because you know at some point you will have a discussion with your developers You do you prefer to write it in in bulleted least or do you prefer to write code for it? And the most likely they say yeah, I prefer to write code for it and then boom so easy you you have automated tests. So Yeah, we use a testing everything that was built and we were strictly having releases at the end of the spring To have release on the end of the street. What's not written in there? We need to have a code freeze And it was very it was challenging at some times, but I think we we got there and we we We were fortunate enough to have a business that could understand that even if we didn't have half a day of development It wasn't that bad as long as we could identify some bugs and fix them and actually have much better and coherent releases so just While I wrap up and I'll show you later at least I can explain some of what we have achieved and with some numbers This is a definition of done document It will Will seem awfully familiar to some of you At the end there, but Essentially what it says is all the steps that need to happen Before a story is considered completely. So that's everything we discussed so far So first thing is to have a spec written and reviewed by another member Of the team to have a test plan written Reviewed and pass But the past only comes later right when the code is complete then We have automated tests written and pass Which you have the code in repository and review. That's the pull request and And the integration testing passing And then you're deploying the study. So up to here You're still within one one branch. So if you if you follow something like git flow, you you still haven't left The brands I mean you might have made it into development and deployed into a development environment But by by this time so you've gone so so much quality already That you don't expect any bugs if if if someone if an external person finds a bag Is an exceptional thing, you know the whole team stops and say wow, how come there's a bug in there? So we we managed to reduce bugs a lot with with this process And then you deploy to staging and that's when you start involving the rest of the people So you start involving the product owners is the user acceptance tests You you involve the client to come back and and see the work But Essentially what we do we safeguard the work in a way So we make sure that when they they actually get eyes on it. It's so so polished and everything's done that They wouldn't have any issues with it Just to bring back some of the specs. So this is a Sable spec the functional part What follows on second section is the the technical part which is more about the the technical implementation So That's the story. That's a user. That's that's a story ID That ties back into pivotal. So if somebody searches ever without ID, they land to that story So it has the user story in the end functionally Has some understanding around what needs to be done and then second portion of it is technical that so you have The there's some detailed explanation on how it can be done. So Again, I think this is very very important Some some people would argue and sometimes you'll find pms that argue that this is a waste of time And you know, you I rather have developers spend time developing and those kinds of Arguments and I can hear them But what I think is important within this is that this is development And this is a more structured way of actually having someone sit down and capture their thoughts of how they will address a development a technical problem problem and Going through that Then they can share it and they can incorporate knowledge from other people who are more knowledgeable about these these kinds of things So this is a great way to upskill your team as well Because if they capture their thinking and then a senior developers goes back and say, you know I see that you're gonna try to create a class out of that, but you know if if you're if you're Injecting objects into the class for instance, and you're not using the dependency injection Then you're hard coding stuff and you perhaps if you do x y or z then it can be more scalable And that's the the thinking that you need that Allows that junior developer to to to get to a better solution. That's easier for you to scale at some point But also get that education that from from the rest of the team. So I consider that very important And this is a test plan so I mean You can you can map that directly into a b-hat test plan In a way, so you have your prerequisites There set up and tear down functions And then you have your tests and then these are steps within the test So in this particular is you select a link you Switch something you go back and you and show specific things are there So it's very easy to take this and if you get into the habit of writing these for every story It's really easy to share the testing burden within the team and make make sure that everybody can pitch in So not having a dedicated team, but essentially everybody can get involved But also we found that this is an amazing way to educate people on the system And allow them to actually learn how the system works and and then train others or Be the help desk and all that stuff. So that was an added bonus bonus for us So, I mean in terms of culture Day one we were an agile experiment, you know, day 180 We were producing interactive software. We had a product backlog with functionality that we could go on for months We have a collaborative and engaged team and we were one team not the client the supplier and consultant and We had an organization and that was really I mean I can't stress that enough. We had an organization that was They understood that they needed to do something in that way To actually get get get value back. So they were supporting us a lot Um So just to go into a bit more detail on the specific kind of results that we saw and Kind of the value that that we've saw we've seen throughout that methodology So as we've kind of referred to within the entire presentation, there was a steep decline in bugs So the processes that tassels has taught us through that allowed us to kind of document what we were creating Talk more about the value and everyone have a kind of mutual understanding of what that functionality was Allowed us at kind of an end result to have a piece of functionality that everyone understood But also then that there was less patchy And less bugs it allowed us to communicate effectively internally to the organization what we delivered And if bugs did occur, obviously we had a process for managing those We've got kind of fully functioning product owner roles and now that project has allowed us to kind of scale that up within the organization So we have an accepted way of working within our entire organization investors and people that we now look at product and platform and functionality And individuals can become product owners and lead on some of those things So they can start to develop roadmaps develop backlogs and work with the development team in that way So we've been able to kind of scale that model Um, we obviously have an engaged development team And from the technical side We are able to use the insights and the expertise that's there and use that from a business perspective So have the rich conversations up front around what we're creating in the best way to do that It's kind of scaled internally within the business our technical knowledge So we know how to better approach how to plan work and what the solution might look like We are iteratively developing new functionality and we understand the process for doing that and we can kind of Scale quicker and deliver quicker for the business and for the value We have more open communication internally our side So not only within the within the kind of delivery team, but we also have that within our wider organization So having clear frameworks and structures for working allows us to better communicate internally to our stakeholders To management or to users within of the system what we've created what we will create What could be created and have more richer conversations around ideas and share sharing knowledge and it allows us to Present functionality that's created and then at that point for example within the sprint reviews Inviting people in and getting their ideas and insights and having rich conversations about the value there with those individuals too Um, and we also kind of have have now a better framework for that strategic thinking So as I say having more open conversations about what value we should be creating what a roadmap might look like And I'll just kind of take you through so for example this this roadmap Looks at the functionality that was created Going forwards we we can start to better plan the groups of functionality We did have a hardening sprint at the end Which looked back on all of the functionality we had created and looked to kind of tidy that up and reflect on on on what we had achieved so Sometimes sometimes that has been very useful for us in the past as well We obviously now have story points. We can better estimate work Which allows us in kind of business side to plan better We know what something might look like whereas before we were asking for requirements and didn't really know what Was involved within that and therefore when it would come or how difficult that might be or actually how much it might cost We therefore have a velocity now and we can better plan Our time so we can say if we're looking at this type of functionality How long might it take to deliver that based on what the team can deliver at any one time? Can I say something on that again? So I think this is the the most typical agile graph So on the first screen we didn't deliver anything And on the second we delivered double the functionality because essentially what happened is the functionality of the sprint of the first spring was sort of there but you know Getting used to the specs and test plans wasn't there wasn't done and but instead of saying That sort of done and then push it we said no, it's not done And we take it up to ourselves realize that all of us haven't done it And let's see if we can do it next time and if we can deliver that little piece of Effort that needs to do the first thing and then move on to the second So that's what you see here, but then you see we stabilized a lot and I mean you always have fluctuations. It's a story point up or down isn't much because That that that stability Helped us then plan and prioritize and actually have a budget estimation through that Through something that's so relative because that wasn't tied back in today. That's just purely story points so we we had like a relative measure that we it was Served and understood between us but that measure Was then used to actually estimate budgets and estimates times of delivery and because we were consistent with that We could do it every time so we we we tied effort into Into money And time basically, but that came organically So that's it, yeah Thank you I just want to add in the back just before we get some questions And that there is a white paper that you can follow on that link and people who follow that link They can get 50 percent discount on the book we have out with past publishing on developing for jupy late So So any questions Please Oh, yeah Who do you prefer for doing qa break managers or developers? and So I have no preference. I mean I think that there's one team and within that team people can be flexible in a way So if if there's time for project managers that they can pitch in The way we work they have a test plan They can do it easily if if a developer is waiting for their story to be Reviewed in a pull request for instance, and they have half an hour they can do the qa and and you know Just have it as a as a lego a lego block There was an interesting discussion we had we had yesterday Mr. Anastasius here And he was saying that you need some distance sometimes From from the project and I agree with that and I think the distance that But I think that the distance a developer has from somebody else's code Is is distance enough to allow them to to test it in a way and that's not the final test step That's the test step before you get it actually in front of of Of your Designer to have the designer review In front of the client to have user acceptance test and all sorts of stuff. So I think I think I think That is my answer basically and I don't know if that answers your question. Well, thanks anyone else No All right, thanks very much. All right Thank you very much for the presentation. Cheers. I missed the very start. So I don't know whether you you highlighted how long it took you to really bed into the process and The build up and the level off and get in tune with it. So essentially it was those two sprints I think so, yeah No, we we were fortunate enough to have very skilled skilled people in every aspect. So I'm not gonna talk about myself, obviously Alyssa, uh, the development partners we had were very skillful And obviously nothing is a is a is a bombless ride We had our ups and downs, but we we were open. We were discussing them And we we were flexible and you know, we we We prefer to fail fast Rather than fail late and retain some pride I mean, we we were encouraging people to to to bring up those kinds of things and we And we we tried to have like a very flat structure. There wasn't any any hierarchical structure. So everybody was able to Bring their their views Maybe One more thing that I think everyone would like to have such an enlightened client Yeah, I think I think the project was kind of initiated our side in terms of we knew that we wanted to better understand the process of delivering that that product and I think A lot of a lot of clients are are happy to kind of hand that over and and not maybe see that process through But our kind of ambition was to be become involved in that So we could we could also benefit from the rewards of understanding what that process looks like and get the technical expertise to to pull that in as well so one one thing to say and that's not covered within this is that then we went and so The the the business Actually, um, we're very fond of what we achieved. So they scaled the agile into other Other places as well. So we did lots of strategic thinking around agile with Impact maps and then that allowed us to to map strategy To goals and all the way back down to development. So that was perhaps that's another presentation For another time, but essentially, yeah I I also conclude that it was a it was really I was really happy to work with these guys because they They were always interested in new ways of working and they were I mean You couldn't ever hear like the negativity. I mean, even if something was off and they weren't used to it they were More in They were more prepared to actually give it a try see if it works rather than just, you know guard themselves in the corner and say no So yeah, I mean they're true to their name I think yeah, importantly, it was a we knew it was going to be an experimentation process Um, so we went into it looking at it like that. Um, and just trying to learn as much as possible Any other questions? All right, so thanks very much for coming and probably you got something out of