 There we go. Can everybody see my slides? Quick thumbs up, thank you Jacob. All right, so what I'm gonna take you through today is some thoughts on hiring, some new hires, what we're doing about it. Some key challenges, and I'll go into depth on one of those and then what we're thinking for next quarter's goals are okay ours. So first hiring, here are the hires that we've made since the last FGU, which I believe is April 13th. So lots of people have joined, really exciting. In particular, I wanna highlight that we continue to be very global, people from the Asia Pacific region, people from Africa, people from Europe, as well as people from North America, which is great. We're continuing to maintain our sort of status as a global company, which I'm really, really excited about. I'll highlight a couple of our key hires. The first is Tom Cooney, our director of support. I've highlighted some of his relevant experience in past companies, obviously coming from Zendesk, which is our tool is a very impressive pedigree and he's already started to collaborate with Lee and Lyle and they're starting to do some great things. So one example of the experience and breadth he's bringing is that he wants to bring customer satisfaction or CSAT back in as a key metric. And so this is something I think had been turned off due to a technical bug. It was not sort of a priority to us. He's sort of teaching us that like, no, this could actually be a more accurate representation of customer's experience support than SLAs. We obviously want to measure both, but that may be one of, if not our main key performance indicator going forward. So we're really grateful to have him and his experience. And next is Jerry. So really excited to have our director of infrastructure here. I've, you can see his pedigree, very experienced, but I'm, I should say one of the things I'm most excited about his background is that he's chosen to make his life in Valencia, Spain. So as we continue to grow, as we continue to get successful and attract some of the best talent around the world, obviously there's a danger that the geography has started to zoom into North America because that's where a lot of startups are or zoom into the Bay Area because that's where a lot of venture capital is. But this shows, I think the power of remote work is that, yes, there are people out there that have, you know, are deeply experienced with some of the most significant startups in the world and that nevertheless are global and multicultural. And I'm really happy that my management team is continuing to be international and reflect the rest of the company. So welcome to both Tom and Jerry. This completes phase one of my sort of personal project of building a engineering management layer. And so I've got a spreadsheet that I track here I've linked to it if anyone's curious, but phase one is essentially my immediate sort of what's called a span of control and Jerry and Tom are the last tires that complete that for me, which is great. And on to phase two, which we have already taken some bites out of but this is sort of completing the management structure in the rest of the engineering function. I already made some hires there. Some of them haven't started yet. So I've got them crossed out. We are waiting for two individuals to have their start dates come up one in June, one in September. And then, you know, we keep growing. We're growing 75% in terms of headcount year over year. So more positions, more opportunities are gonna be opening up. So I've got phase three in here but who knows what phase three is gonna be but we're continuing to track it and make sure that we're ahead of these needs so that teams don't find themselves without managers so that managers don't find themselves are burdened with a number of direct reports very much paying a lot of attention to this. This is what that sort of span of control looks like. This is a slide for my engineering 101 for new employees but I figured every once in a while it's good to check in with the whole company so everybody knows what our structure is like. Of course, it's also reflected on the team chart which is in the handbook in public to the world so you can check it out there. But these are the sort of eight departments that comprise the engineering function and represent my direct reports. And so we're starting to feel complete, starting to feel symmetrical and starting to feel like we can steer the ship which is an important thing. We're a large department, we're growing, making sure that we're all aligned requires really keep people in important positions making the most important decisions for their group and collaborating with me and their peers. Our most important vacancies so SREs and the production team we've made a couple of good hires recently not just Jerry but Devin who started and another SRE is gonna start at the end of June in fact a former Facebook SRE which we're really excited about but I think we have probably three or four more to fill on that team. So that's one of our key roles. Then we got a engineering manager position on our database team that's been open for a while and need to make progress on that and a lot back in developer positions open particularly the monitoring team is the smallest one. We really want that team to gain critical mass and what I'm calling escape velocity where we need a team of six or seven by the end of the year and we need to really build around the bend and he and Dalia are working on that but please refer anybody that may be your network that you know. Challenges, getlab.com ability and performance still feels like something that we just don't quite have a handle on. This is for sure one of the things that Jerry is here to own and help us with but lots to do here. One of the first big steps in that is our GCP migration project. I've linked to a chart on the right displaying our burn down which shows particularly the geocomponent of this we're in sort of product development mode. We're solving a lot of issues but we're creating a lot of issues as we encounter unknown unknowns and that means the achievement curve is essentially flat, the burn down is flat. So I literally just left a status call about that meeting and the team's working on what do we do to fix the slope of this curve and eventually get to a date that we can be sort of confident with. I've linked to the status doc for that call here if anybody wants to read the discussion that's going on in real time but obviously something that we're spending a lot of time and effort thinking about. You know, I talked to, you know in the prior section I sort of talked a lot about hiring, bragged about a lot of the hiring that's going on but there are hotspots in the organizations all for various different reasons where the pace of hiring is not matching the overall pace of hiring. So we kind of look good in aggregate across the function but if you zoom in at specific points there's various roles or various departments we're sort of under hiring and I'm very focused on those and helping with those. And these are some of them. And then clean interfaces which I'll go into in the next section come drill into this challenge and what we're doing about it. So by clean interfaces I'm talking about our org structure our team names and to some extent our titles, our specialties and expertise. So we wanted to make some changes here. We wanted our function to be easy to understand. We wanted to simplify the active release planning that product managers do. They need to understand like how much bandwidth do I have? How many people do I have? And we didn't have clear answers for them until very recently. And then of course we also want to maintain functional managers for our front engineers and our UX designers. What a lot of people do to solve this problem is just verticalize your team. Have a team that contains a front end person a back end person a dev outs person, quality person and a manager and you're off and running. And that's a great way to remove a lot of gates ease communication but then things like career development and things like hiring for specialized roles really goes down the tube. And we know that from our past experience we know that from our peer companies. So we want to maintain the structure we have but we do want to simplify our interfaces and this is quite important. So the sort of concept that we're all snapping around when I say all product development which is not just engineering but also a product management is the DevOps life cycle. And so you've seen this diagram likely on our product categories page I've linked to it here in case you haven't this page does evolve quite rapidly. So I encourage you to check it out at least once a month. It's owned by Sid and the product managers Mark and Yop are our product management leaders. But this is the sort of concept they're based on. So you're going to start to see this reflected in team names and titles more and more and more. So we said that individuals and the front end UX teams are going to get specialties that are attached to their titles on the team chart. And it's going to reflect these what we call DevOps life cycle stages that are going to look symmetrical with the backend teams. The backend teams have them in the team name other individuals have it in their sort of specialty and this is going to eventually be symmetrical with PM. So obviously both departments are growing like crazy so it's never 100% in sync but now that we're aligned on what it is we're doing here you're going to see them be reflected more and more closely. This will give PM dedicated people for planning purposes so they'll know I've got this many front end people this many backend people this many UX people and that's going to make their lives significantly easier and hopefully cut down on a certain amount of project management that our product managers have to do today allow them to focus on quality. And then all this will be public in our team chart so transparent within the company but also at the outside world. So this is shipped for UX design. Sarah picked this whole thing up really, really quickly I was able to ship something I think earlier this week and most of it was done last week. So you can see a screenshot of our team chart here and you can see a link to the depth where we put this in place. So you can see for example, James on our platform team knows that Chris is this person for three UX design and so forth. And then for front end, it's a larger department there is a little bit more complexity here so we couldn't SIM ship so to speak the same thing but we have a similar pattern in mind. Tim is working through it and talking to his people and product managers and what we're hoping to do is ship something next week. It looks like we've got something that's really good that should make everybody happy. We just can't quite merge it yet without a few more conversations. So that's what we're doing about clean interfaces and the next thing is OKR. So again, we're limiting ourselves to three for the benefit of focus. I've linked to the issue here where we're authoring OKRs. We've got about three weeks to figure out so it feels like plenty of time to set some really good goals for the next quarter. Of course we still have flexibility as business conditions change, as technical things change, we can change throughout the quarter but I really like to kick the quarter off with solid goals and we've got a perfect amount of time frame to do that. We're going to do some, of course we're going to do hiring OKRs as we always do hiring is really important and we need to maintain focus on that. We're going to do some new things this quarter. We're thinking about error budgets. If anyone has read the Google book about site liability engineering they have this concept of error budgets and we're trying to figure out how to one do attribution at the team level not the individual level but not the director level of the team level for issues either outage or degradation like gilab.com and then how the individual teams can spend that budget and different companies do it different ways so we've got an issue where we're openly discussing how that should work. We want to incentivize the right behaviors so we don't want to encourage gamesmanship, we don't want to encourage slowing down or stopping shipping or anything like that so this is a very important one to sort of word correctly and attribution particularly where we've got a large app sort of a monolith can be tricky so we're going to do an iteration in Q3. And then the other thing I've posed is hey maybe we take prior to having new continuous delivery tooling and things like that what if we just take the current method that we're deploying to gilab.com via the omnibus installer we use for ourselves as customers and just start doing it weekly as opposed to monthly by thinking being deploying smaller chunks of code whatever that is is better than bigger chunks of code and it will help with attribution and it will help with forensics and other things. It's by no means easy though but I think a lot of people are up for that so we're kind of thinking through what would this mean what do we have to do is it possible and what exactly what the benefits would be and we have until we kick off the OK or just to kind of figure this out and if it looks like a good goal we're reasonably confident we'll kick off and we'll figure out the rest kind of interviews we go. So with that happy to answer questions let me stop sharing and I will open up the chat. Let's see yeah lots of comments about Tom and Jerry. That may be too much of an American reference I don't know if that cartoon goes globally but yes we do have directors named Tom and Jerry and we have a Tommy too. Let's see. Let's see. Brennan says I love the engineering team is a friend of this is clean interfaces great analogy for those who understand it. I think this is actually a company one thing this is something that Sid is quite passionate about and it's multiple places in our handbook so we can't take credit for it but we are kind of channeling our CEO and he believes that naming is one aspect of presenting clear interfaces and that it eases a lot of communication so we're definitely trying to take that to heart and to reflect it. Let's see Tommy says usually about 30 seconds change 30 seconds after change goes live we're still accurate really appreciate the attention that goes into the product categories page though it makes it really easy for Dalia and I to prep for candidates. Yeah this is an issue where the product categories page has been changing very rapidly and then as on the engineering side we're supposed to be reflecting it when it's who's on what team what teams are called people's titles obviously we want to make sure we're communicating that delivery and people don't feel like their heads are spinning so we're trying to figure out the right way for engineering to take part in those discussions subscribe to changes for that page just so we can all kind of stay and stay in sync don't want to slow down we just want to make sure that we're staying in sync and hollow line. Clement says dang UX teams iterating fast yeah agree they're moving really fast which is great and Sarah's got some cool plans for Q3 which you'll see in the form of OKRs Bob says I think it's worth mentioning that every issue discovered in GCP staging fail over an issue that we can avoid in prod so we're not standing still in the migration correct yeah this is all really valuable stuff to be finding but because it is unknown unknowns it means our predict our predictability is quite that and everybody is very curious customers executives internally when is the date when is the date and it's a challenge to provide a date and provide confidence in that date when we're sort of swimming in a river so to speak so we've got to find a way to slow down that river or speed up our swimming so the trend and that burn down is an accurate fit to the curve and right now the curve is very sort of flaky and linear so we got to get a handle on that not shipping it or shipping it slower isn't an excuse but by all means it's all great information to be finding these are things that would cause issues in prod and it's great that we're sussing them out through the act of rehearsals or whatever called agile demos let's see Philippe says release weekly our deployment is going to be predictable same day same hour that's a great question I would jump in the issue and pose that question Philippe you know to me this is an iteration towards continuous delivery where small bits go out when they're ready in which case you kind of care less about dates but obviously that's a big leap to get there and so yeah we may want to do something that's like on a given day and you know one of the goals of this is to decompose the big monthly feature freeze but it does mean maybe we have to have weekly feature freezes or something like that because clearly a certain amount of manual QA QA needs to happen and certain people need to be around and so yeah it's likely that we would probably snap to a day or something like that maybe we care less about the hour and that's a good way to sort of like train and prep ourselves and build the habits we're going to need for true continuous delivery where it's like things go out the moment they're ready and ideally the production team doesn't even know in codershipping because things are so smooth but it'll take a while to sort of get there but I encourage people to jump in that issue and let's get everything on the table about what it would mean to do that. Yeah well Eric if I may. Sure. What I wanted to point out if we have a feature that is split on different merge requests for example I don't want to be in a position when I merge the first merge request and it depends on other code that is not deployed yet and we have something working in production so if I know that the release point is going to be in one hour I will wait until the production is deployed to merge everything. Well another thing we're talking about recently and Dahlia sent a merged handbook change and is communicating this out today and next week is we need to get to a point where master is sacred and pristine and everything on master has proven to be deployable that's the heart of continuous delivery. So for something like a merge request and another merge request if there's a time gap between those you've essentially broken the build and we need to adopt a culture we never break the build and that means integration branches it means perhaps a staging branch prior to master where things can be proven out in a specific environment and we need to get there I think that's definitely a part of this. But if you have a feature with front end and back end if you merge the front end then right away it's going to be in production and after I click on the button to merge the back end we have the front end and not the back end in production. So that's this very thin line I want to avoid and make sure that we're not in the disposition. I'm adding more. Sorry go ahead. I know you've had a lot of thoughts on this. Sorry just real quick we are moving toward the CD vision and part of that is also talking about how we could use something like feature flagging we have to get to a place where we're able to do small merge requests that don't break master and one of the ways is using feature flags there's obvious things that you need to think about we don't want to end up with thousand feature flags that we have to manage. So I'll add that to the issue and we can discuss there but I think to leave that would help as well. Yeah I think feature flags can do this I think integration branches can do this I think keeping master peer can do this I think also having a separation of concerns between front end and back end we have a back end API that can evolve on its own and then a front end API that can consume it that would allow us to shift back end changes first front end can come on and consume it after the fact without breaking it and if the API is sembered that's when we're able to manage these changes as well. So lots of different levels which can handle these things and many tools in our toolbox to do it. So Erika I don't I hear you say staging branches a staging branch I don't think that's that might not be where we're going. I think what we want to do is continuous delivery so what GitLab allows is to have a staging environment for every branch and to review apps. Yes exactly. So it's more about those review app environments than that it's about adding a lot of steps in before I think that's the old way of doing things not the new way. True yeah and Mech and I have been chatting about this and that's one of the OKRs that Mech and his team and specifically Bremen are thinking of taking on is review apps for every branch in the GitLab project which would be a huge step forward. It's a great point to say thanks. For sure yeah and feature flags for sure and not for everything but for if we know that there's a likelihood of changing if it's a big change and it's easy to add them then that makes sense as well. So Philippa who's one of our current lease manager says it's possible we've been doing it already. Yes but as we're talking through this we realize there's a lot more to do to kind of do it in a true sense but I'm glad one of our lease managers in particular is enthusiastic as well as dreadful about doing something like this because it is a big change. And let's see. OK so Sid a link to the handbook change stable counterparts I believe that's the handbook reference I was looking for where interfaces are mentioned so next we're posting that. Yeah we don't call it interfaces because interfaces is pretty generic. It's about your counterpart. It's about the person on the other side and it's not about them being clean but being stable so they should deal with the same person every time. Obviously it wants to also make them clean because otherwise no way we can get them stable. It's just very simply dealing with the same person so you get to get to build a certain trust and certain credit with each other which makes all communication easier. Cool. Let's see Dan says sounds like a great feature track prod errors to teams so we can predict potential problems and release planning. Yes it's a very time tested sort of pattern but difficult to implement in practice without some work but we definitely want to get there. York has some ideas about well if we can't do you know line of code level attribution database tables are a way to start so we're looking for our kind of first iteration to get better attribution and that's part of the issue but if you have ideas if you have experience jump in there and we'll figure out what works for GitLab because this is different at every company that tries it and experience. Let's see John says for error budgets and attributing problems to a team I'm worried that we would develop too much into blaming how can we avoid that. Yeah great point we definitely have a no blame culture this is about being better not about casting blame. This is one of the reasons why we tend to focus on team level attribution and never individual level attribution and I think it's you know it's just something I need to keep reminding people of that like we want to be better each day and we're not you know seeking out problems. We definitely want to hold people accountable but that doesn't mean shaming if that makes sense. Yeah Tommy says if we let it turn into a blame game we're not managing correctly. Yes I would say you know look to the management structure I just talked about to prevent this from happening if it's happening it means we are not doing our job as a management team so hold us accountable for that. Tommy talks about understanding and mitigating risks not about apportioning blame yeah. Yeah and so like one of the things that well one of the consequences of a team you know going over the error budget is like well they'd have to spend more time on QA or something like that. So it's about dialing them in so that they're shipping fast. Like you know I really want to avoid the phrase like slow down or something like that because that's not what we do that's not our culture but dialing in the team's calibration so that they're shipping with the appropriate amount of quality but still shipping as fast as possible it's important. This is something that mech continually beats the drama of is that quality is about moving faster never moving slower. It does mean spending time on specific things but at the end of the day when you're spending a lot of time on forensics you're spending a lot of time fixing the same bug for the third time that is wasted velocity and that means shipping slower. Sometimes those things are hidden in your numbers and we want to make sure that we're always shipping faster and slower. Let's see, Sean says front end and back end MRs are an anti-pattern. Yeah, I agree. I think we can find a much better way to do this. We've got like I said a few tools in our toolbox to do this better. Lots of comments about that but so by all means jump in do we have an issue about that? I think this is maybe a level of detail underneath the weekly releases thing. Maybe we should have a separate issue to discuss that so I'll get that started and I'll email it out to everybody that was on the FGU and we can move the discussion there. Or I'll just slide it to the development channel. So anybody passionate about front end and back end MRs I'll post an issue after this to the channel and we can have a discussion going there. Let's see. Excited to see movement towards GraphQL. Yes, APIs in general are great. GraphQL in particular is an exciting technology for sure, it's from Andre. Bob says we have underscore version files for pinning versions. So let's say this is a lot of excited talk about front and back end MRs and you know, changes and things like that. So let's get all this into the issue. As long as we get the release train choo choo. Yes, release trains are again, like you know, release trains to me kind of speak of handoffs and gates and things like that. And so I want to try to minimize that release trains. I come from a company prior to this that was doing hardware and stuff like that. And there was some true waterfall going on there and release train really became a dirty word for me in that experience. And so when I hear release train, I'm like, oh, we got to rename that because we want to stay agile and we want to keep moving. But it's, this is about an automated release train. So not about the people all putting their things together. You just press the merge button and get a release train. And it's to make sure you can release faster than your tests are running. So if your tests take 50 minutes, the release train allows you to deploy every less than 50 minutes. So maybe we could call this like a release monorail or some kind of driverless train or something like that. That would make me a little bit more comfortable because the release train. There's a feature proposal for it. Feel free to make it release Mac lab. Let's see, Kathy says holding teams accountable should definitely not be perceived as blaming, shaming, right. Yeah, so she, you know, for Kathy's world in security, there's a ton of learning for us to do and security moves so fast. It's almost like a war of attrition. When we find security issues in particular, it's like, hey, this is something that we need to learn and obviously we don't want to repeat mistakes over and over and over again. But every week there will be some new technique or some new vulnerability. And so avoiding blaming, shaming is important. Everywhere particularly on security. So I'm glad she holds that value. Could a product feature help with error budgeting? Yeah, I think so. There's potential things in monitoring. There's potential things maybe on the victor's side of the world where we manage our projects in the app that could help with this. So for sure, let's get some feature requests and emanating from engineering about what we need to manage our product and our process efficiently. Will we be iterating on enforcing quality and master? Do we know what the next steps are on that? Yes, we will be iterating. I can answer that for sure. Do we know what the next steps are? And I think that's up for the current debate. I think we should have that discussion and issue about weekly releases, about what enforcement means. And obviously with an eye towards moving as fast as possible and avoiding the accountability problem that's been talked about. Josh, link to the review apps issue, great. Everybody check that out, add your thoughts there. I think that would unlock a lot of productivity. Let's see, Tommy's talking to Flippa. Release Rocket, yeah, I like that one, Clemente. Let's see. Okay, now we're getting into jokes about the release stream and release rocket and release monorail and stuff like that. Someone says Daniel has figured it out. The release hyperloop, there you go. That's the final answer. Cool, any other questions, either verbally or on Zoom chat before we break? Going once, going twice, sold. Fave ice cream, chocolate, obviously. Neurotransmitters and Dorfins, it's not just a flavor. Cool, well, everybody, thanks for the feedback. If you wanna get the discussion continuing, join me on my VP channel, jump in some of the issues that I linked to and let's keep this going and I'll create that new issue for the thing that I talked about earlier. Cool, all right, we'll have a good day everybody and see you in the team call, cheers.