 Okay, so this presentation would be about estimators of the lost arc You can see the Indiana Jones reference and it's the story of the advanced reconnaissance group My name is Michal Koneczny. I'm the mage from it is monitoring.org. That is my official role now joking my official role is senior software engineer that had and Who are you again? I've never met this man before in my life No, my name is Ethan lonely and I'm the product owner for the community platform engineering team, which I work with Michal and other majors other sorcerers in the And yeah, we're going to take you through how we came about creating this team within a team Founding member sitting right here. So I need to be very accurate So today's presentation like I said is going to take you through the origins of the arc team Where it came from where the concept came from where the acronym acronym comes from because that one What the purpose of the team is for what we use the team for within the wider CP team It has its own heroes and villains as every every good story does and then we have live inaction tales from the quest that this team and backs on and I will tell you through all the battle stories and then we will unveil our chest of treasure that we always find at the end But first I'm going to take you through the origins of the arc team So At the beginning there was only a very small spark This spark grew into a fire and the fire started and people were terrified of this fire They really didn't like it at all But in the you know, they started warming up to it and Realize it might not be that scary and it could be a little bit helpful and the fire is absolutely a euphemism for agile So our team started in this agile transformation journey and we wanted to start Like planning some project work and delivery timelines and you know, we needed we need the requirements And we needed deliverables and we need a success criteria And we were going to get all of this stuff and then I was going to come into the team Oh, there was going to be somebody to take this stuff for me And we were going to pass it to a delivery team Michael and others and we were going to have what people wanted at the end Like that was going to happen like I mean how what what else is needed. I tell you what I want and then I say great I'll give you what you want and then Yeah, you don't get what you want So this was our idea that we were striving for like holy grail and we were going to get there. Why wouldn't we get there? Really really straightforward simple Also, we thought so Problem that we had was while we were getting these requirements and these asks from the questers and You know, they were there that we had our success criteria. We knew what people wanted Just hadn't a notion how to actually deliver it to them. We had lots of or rather We knew several ways to deliver the thing and we couldn't really decide what is the best way forward and Maybe familiar with when you're tasked with something. There is a some indecision as to what technology is best to use and in the in the project like Fedora that is So complex and wide and cool, but like there's a lot going on You know, if you don't choose the right tech stack at the start It's very tricky to know how many other people's projects are impacting if it tends to be something It's not compatible or there's dependency issues or whether you're going to choose something that will lock you in in 369 months down the line when you've just developed something else So our team was wasting an awful lot of time figuring out what the best path forward to be even begin the development was and that was causing you know It's causing a lot of Obsessed would be a nice way to put it within the team They were frustrated with me because I wasn't giving them clear enough requirements I was getting frustrated with them because I wasn't sure what they were asking me for it begin with I told them That's what they wanted. They want a red button. What do you mean? There's different shapes of buttons. Give me one So like Yeah, yeah, pretty much So there was there was a lot going on and yeah, like and nobody knew what we were making it up as we went And we were delivering things, but oh my god. It was not a comfortable process And this was happening for a couple of projects happening for a couple of months, etc So in the end We said it up enough. We have Too much going on. We need we need to cut a break here And I was fortunate enough to like have Pingo as a as a friend and confidence and also a team member who's saying What if he's going on the project even they haven't started yet? I don't know what to do So we were discussing the problems and he did come up with the idea of like, is it the technical prescoping of the problem? Yes, I don't have anyone to like vet this with You know, I can I can vet it against like business requirements and benefits to people But I have nobody to actually vet that this is not going to cause a dependency in the library over here That's upholding All of release engineering. I don't know, you know, I didn't have that context So we decided we were chatting and said why not propose a solution? The solution being we pre-sculpt work technically So we still get the requirements from the the request. We still get the success criteria We still get all that stuff But before we even say yes or no, we pass it to a subset of our team And they investigate whether this will actually even work and if it doesn't work, good luck But if it does work, then they're going to look into like how we do it the As trade-offs like, you know, you're going to grab the golden idol and you're going to have to put something in its place This the trade-off was that I would need people from my team to spend time doing that And that means taking them out of existing project work So the the barter was We dropped projects we had been taking on like I think there was four projects running plus our sustaining team as well So we had five subsets of our team running which is full capacity. Nobody would left over you took a holiday things stopped I'd like to tell you it's completely different now, but it can't be a little bit like that still But we really had zero capacity for Innovation or thinking things through So the trade-off was drop projects drop everything bar one and put more people into our infernal entity And they will be able to then take some time and pre-scope this work and work with me once it comes from the requester to us And we'll work together and we'll better technically give us some options And then it's ready and then the development team can go So that was the solution send out a comms to the managers send out comms to Whoever needed to know and they said yeah, okay, cool. Let's do it. So we did it And here we are talk about it because it worked And we had a lot of fun then with the acronyms. So like there was a good deal that came up Obviously the advanced recon crew is the one that actually can be publicized But like there's there's a few there that was um, that was a hot contender The angry rabbit's committee nearly ran nearly one Um smooth wanted either rating children No, we said let's keep a professional for god's sake Let's just try and be like adults. So we've called it the angry con or the advanced recon We stuck Indiana Jones So anyway, so that's how the hat set started that was our problem. That's what we were facing That's the solution we came up with and it really has been By far the best decision. I think we've ever made in terms of product delivery within the CPU team And I'm gonna fight any one of the degrees And Mikhail will take you now through the rest. Okay Not through the rest, but through few slides Okay So first every good story needs a hero. They emerge from the battle and scared They were heroes that everybody talked about. They were the ones whose enemies tremble when they hear them coming but who they are really and It comes to me that these are the heroes of the it is story The arc is actually made from people from infrared launch team and they are the heroes You can see some of them here Unfortunately, we don't have these trade cards for everybody and the team yet, so what we will Need to thank design team ferro design team to making tools So few people here We usually just try to find the best suite for the investigation for the other organization And it depends on the on the thing we need to risk to investigate And every good story needs a good willing The people were scared when they saw them approaching There were nasty group and didn't bring anything good But they were still imposing figures And who are the villains? Oh the girls Okay, so One of them was having the clear requirements. We usually get some request with some requirements, but those are not technical or they are just something somebody wants So one of the villain is the requirements. We need to actually get to him and defeat him The next time is next one is time And time is always a villain every time and in this case we didn't had clear estimate We were struggling with the starting project and When we thought it will be ready, it was halfway through. So we actually Trying to do this during the arc investigation to estimate time Who how long this will take it's usually a good guess or a good estimate because The people are trying to Work on proof of concept trying to do the work that is we are trying to investigate so it's It's good usually good estimate and we are delivering on times which says a lot And third one is technical options This means that we can see in the arc investigation the options that we can think of And the team is actually trying to explore them just trying to look at them and see what what is viable And we prepare for the quest So first we get the underpants all the underpants we can find Then we do magic and then we have profit. We have a profitable business at the end and this is where we do it No, I'm joking But yeah, we would We would rather hold the underpants quickly If we do that, so we decided to go all the way First we will choose the crusade. This is the arc investigation We actually want to do and we will We will actually explain why this is needed. What is the story behind it? Why we need it Then we choose the right heroes as I said before Their heroes are chosen usually on the arc investigation if this is some old Project we have an event to rewrite it We actually want to have somebody in the arc team who knows the project who actually at least have some knowledge about the project and If it's if it's some specific technology, we want to try we want to ask people to be in the arc team who have some knowledge about the technology So we have somebody who actually knows what he's doing in the arc investigation And then we go to the adventure and they left for a great adventure Looking forward to see what they will found what new beauties they encounter But happy moments they experience, but it will not always be a positive situation so We had a plenty of investigation. I in this case open up Okay This is why I'm wearing the glasses Let me just do it little bigger So these are actually the Arc nodes with this is the output what we do. This is the Output of the adventure these are the nodes the journey of the heroes And here you can see what we actually What initiative we have and what we actually did for the investigation We have plenty of completed that are not implemented yet. We have plenty of Implemented as well And as was said before We don't always have Everything is not always great so we actually encoded few of the Things that we needed Sometime and it was a great that this was actually found out in the arc investigation Not with the project actually working on something so For example, what do people use FMN for when we try to rewrite the federal messaging notification system We're looking at first What is there then where they were plenty of customer rules for plenty of notifications And we wanted to ask community first what they actually use And what could be useful so we don't run In the same case like we have database for with plenty of customer rules That are used by one person So we want to share them We got Some examples we still even now when the FMN new FMN is deployed We still have some request for some Messages that weren't there Next thing is how could we scope badges properly? This was a big one because the badges are here for a long time We don't want to take the maintenance ship of badges your team We want to leave this on community But the badges needs to be written and we want to help with that So in this case, it was how to scope the badges properly and how to actually Make it work. So what we can do how we can design it to be better And the last thing is Is for the Activity count script that is new and There is still ongoing discussion. What should be marked as contribution in fedora because How you can recognize contributor by contribution, but if they are not clearly Defined it's hard I can actually show some of the other of the stories For example the good example not everything that is bad Webhook to fed message when we did the investigation We finished it. We had some design of how the webhook to fed message should work And then we look at the github to fed message and found out that we actually designed it in the webhook. So That is when they just the second investigation was just Writing all this should be implemented like this Next thing that I can talk about Michael before you do can I just point out as well as any project manager in their product owners or whoever in the audience If you don't have somebody looking into that and you pass the development team Get up to fed message and they do the work and then you say okay Next we're going to do webhook to fed message and they go the same thing like your credit your credibility shot Like it's just it's a waste of everyone's time. So like Highlighting that as part of this art process was just huge because we were like great That saved a huge amount of time and it wouldn't be possible without somebody doing that free scoping work too from a technical perspective yeah I would say a second Second thing about this or second example at the end The community shift is now implemented It was stuck at the first time for a long time at some law issues mainly on the gdpr stuff and This was something we couldn't do anything about and it was actually something we Did arc investigation for but we didn't look at the law Law requirements and this was actually met When the federal legal team was actually started looking in it. So yeah, not everything could be catched by the arc And the last thing I would like to talk about is That we have always more than one way So the arc team is usually looking I can open one of the investigations so we can see It was a fun revelation to work in software with software engineers to be like. Yeah. Yeah, but there's all the different ways to do it You can see List of the ideas that actually were looking up in the in the case of datanome data grab And all of those were looked up by the arc and said no, this is not Well, it should work Or Yeah, this is a good way to go forward and there are conclusions And recommendations so the team who is actually starting on it something to work on work with And Give it back There we go. Yeah fortune and glory kid. That is what we were doing it for No, and we were we were looking forward to the reward that was waiting at the end because we were going to be Guaranteeing some technical clarity like who doesn't want that and we were going to have guardrails for development teams to pick up the project and work within Not rules, but like Rails that they wouldn't be worried about stepping over But they would have the confidence and they would have the the surety of doing some development within And without going like way over or making ridiculous choices that they didn't understand the ramifications of that far So yeah, fortune and glory But there was there was there's a hell of a lot more to it than that The real fortune and glory that we have actually seen within the team Is and hopefully as well, I would like to think of the core project as well Anything that the cpe team has put out in the last maybe Two plus years since we started using this arc concept is better services We have upgraded the thorough messaging notification system based on arc You know, which is going to see your thorough messaging notifications be a lot less laggy You might get them on the same day that you have that the thing happened, which is an improvement You know, we will have a We've we did some work updating Bodie. We have done nomodatic refer There was there's loads of services that we have updated Based on the outputs from the arc team as this is the best recommendation So we better services back into the community. All right And our development team was getting more clarity on their work Because it it can be painful to to do some of this heavy development stuff So it's it's nice to be nice And if we can be nice to the development team by giving them a little bit more clarity and structure Then they'll be happier to do it and you don't want to piss off development Sorry for swearing but you don't want to piss off their engineers because people deliver the projects So treat people nicely and the projects will be nice And you're going to get less confusion from your development team We're going to get a lot of less time wasted at that up front trying to figure out what path is best Because people have already looked at a couple of different paths. They've already been able to advise you on this is probably your best shot here This is an agnostic enough service. It's not really going to do a lot of damage. We roll back etc So It's easier to just pick up and go And as Michelle mentioned like we have had successful deliveries of all of the projects that we've picked up Anything that we've actioned we have delivered on they've been successful And we've also kept them to a very good timeline, which has made it easier to plan around as well So for people sitting in that project planning management that kind of that kind of sphere It's wonderful to be able to give a very good guesstimate of how long something is going to take How long you're going to not have team capacity for or how long you're going to need capacity for as well And you can plan it better and you can start building out like Short-term meeting term long-term roadmaps, which will get a little bit fuzzier the longer you go But you stand a hell of a lot of the better chance planning properly when you have that clarity already on the work in your backlog Speaking of backlog you get one properly with delivery plans Whether in our case it is initiatives their projects It may be like epics in your backlog, but by having a team within a team like an arc team Doing those investigation spikes pre-scoping the work Forgetting your backlog slowly scoped and it's getting delivery plans Already built in so that by the time your team has capacity to start picking up that work You have ready to go and choose from at the very most you're spending Maybe I think we've spent maybe a week five days Just reviewing the arc documentation and onboarding into the project But that is it we've gone from like three four weeks of Oh crap, who should we be talking to about the service to just take them out going? Hi, that still makes sense and away they go And uh, very importantly we have an opportunity for community feedback in the projects before they even become long Which is extremely valuable obviously for the community platform engineering team And I referenced the fnn project most recently and I'll credit ryan lurch for doing the the bulk of this work and before we started the arc investigation we were compiling the requirements and You know the use cases were so vast and so many to try and hover and ryan just mailed out to the community and said okay What do you use the service for and because people coming back to him with their their use cases He was able to compile kind of like but this all fits in this general buffet and that should be covered by this Rose we don't need all of these custom rows that they've made to virtually get the same thing These are a bit more use case Corner casey. We should probably check if they'll be covered by something else or whether we need to keep the customer But it then it meant that like we weren't spending development time Stripping out all of these rows from a service only to launch the service and have a load of people going What the hell I have I have my think and figure to this and now I don't need more I'm like, I don't know if that actually makes sense. It did because we had already pre checked it So that was extremely valuable to have So yeah, it was great to it was great to have that feedback loop And as you saw the documentation it's open, you know, you can love on you can see you can read through all of the other ones The ones that aren't implemented Please feel free to read them through and submit your other messages and say like You know, actually there's a better way of doing that. We're open to that. That's why they're they're not implemented If they're even if there's a draft Throw in your thoughts and ideas there We want that we want the feedback We are always Open to know new acronyms. Yeah You can open pr for that as well Okay, and we have some numbers because I can read numbers We have 70 and finished arc investigations We have seven projects that were actually delivered with clear requirements and on time And we have eight acronyms suggested for arc And this could still still grow Okay, and this are the some links so we actually had some issue uploading the Uploading the slides, so hopefully we will solve this and Here are some links. So Initiative report. This is where we have the initiatives Arc investigations. This is the document I showed earlier earlier And arc repo is the sources for the documentation And that is all So, thank you So you have a lot of people in your team work from my perspective Do I have a lot of people in your team? How can the same concept be applied to smaller teams where you might have only three people working on like three or four different projects? And you have to vet those projects so you're not wasting stuff too much. I want a small Do you want to take that? Yeah My suggestion would be you need you need buy-in from like your stakeholders and your management team to actually Not separate your team Maybe there is a heat there is maybe one or two people off that three or four that can do that free scoping work If you really are only a three person team I would try and get buy-in from your your manager stakeholders to like We just call it a spring zero to set aside some time to like investigate that properly It depends on the size of the work Some of our investigations have been finished in like a week and a half Some of them can go on that maybe like Five six weeks You don't try and keep them any later than that because otherwise it's a project and that's not the point But um, but you do you do need buy-in for that free scoping and if you're a small team It's probably going to fall to you And you need to be pushing the the wasted time that you're spending or build that into your your actual delivery plan all together Yeah, because we have a lot of campaigns and they're all hyper focused On their specific app and there's not much overlap between them So maybe there needs to be more overlap If they're all delivering up Yeah, maybe you do need to like pull something from each of them and get them into a room and just get them to hack it out Like for a week or two and then come back and say, okay, deliver this deliver that one Liver this me to the end Yeah That's one of the first things we did Yeah The art team is never the same people all the time so if you're I've misunderstood a bunch of men who are just one team But if you're many teams in a kind of a bigger team to just step first You probably do need to look at like forming a standalone team like that Like we were fortunate to be able to have our infrastructure and arrange team that figures support sustaining work But that's where we pull our our investigation team from and they're never the same person There's there's definitely like there's a request to come in of all different skillsets, but You need a variety Different people talking about And just to make it clear we have in in fun ranch team people from centers infrastructure from federal infrastructure And from federal release engineering. So we have people from all over the place and They work And developers. Yeah The mix is For this is Do you set kind of guard rails for the art team if you mentioned you do group concepts And I find it very easy for your group concept to sort of just become the concept. So do you kind of have rules of that where you stop the art team? They they stop short of implementation No, they're usually you know, usually we have only Few weeks so it will take four weeks. We are in some dead end and we shouldn't go this way. So But there there is never only one option Explored so there there is a rule that we don't Just look at one way. We do and the team does investigate At least two if not three ways Some will be automatically ruled out, but they will still be looked at like I think we were looking at Replacing the off-lead library And there was one that was absolutely not going to work for we were still going to look into it to be sure to be sure And another option was but they were both looked at both put forward and then you know, you recommend Yeah Things that have taken like the Around the five week mark has been things like the fnn project where there was time needed to get people to come back to us with answers on like what they use cases are the badges project is also falls into that as well that people need time to like To think about what they use these things for and why it's important to come back with us And we found that important enough to give that extra space because we didn't want to then start looking at our at our options With upon a view or with only half the information We try and get as much information off from before we start out So that we're we're looking at it as in the most holistic fashion possible And we are okay to keep this little bit extra We also have a new way to doing it And that is if we are working on a ticket on a regular ticket that we get just in the federal info range and taking too long we actually just Uh fill our fill in the arc investigation and leave this to our team dedicated team that the world Yeah, so anything that comes in that's too big. That's like this should be a project that needs more people I don't know where we're gonna take this goes into that side of the world And we pick it up and look into it and see yeah, okay, that's big enough We need to And this project I use this in this case in the arc investigation is actually done by somebody who tried to implement the ticket and didn't actually I finished it But at least started Any other questions before we say goodbye Thanks, pingo for creating it To work. Yeah, but anyway, yeah, so that's that's the arc project if you have any follow-up questions or anything you can reach Leo McHale our emails are listed at the bottom Or you can find me at the bar later Oh All right, thank you very much