 Here we go. All right, so the next session we have Brandon O'Leary, who's a new product manager for the verify team. I mean, Brandon, I think you're a longtime veteran of GitLab, but you're recently enjoying the product team. So I'll let you introduce yourself, and I think you'll probably have something you want to share on your screen, and I'll let you kick things off. Great. Thanks, Ray. Appreciate it. Yeah, I'll go ahead and share my screen. I just put a couple of slides together, hopefully not kill everybody with slides, but helps give context to everything that I wanted to talk about today. So hopefully you can see those now, yeah? Yep, yep. Great, great. Yeah, so like Ray said, we're going to talk about the verify stage of the DevOps lifecycle, and just to kind of introduce myself. Again, my name is Brandon O'Leary. As you mentioned, I've been in GitLab for a long time, which is October 2017, but that's pretty long in GitLab years, and recently made a transition to take over as product manager for verify. And so what is verify? I was going to kind of talk about that first. So if you're not familiar with how we look at the DevOps lifecycle, this is kind of the way we think about it, and we think about it in these DevOps stages that I'm sure you've heard about, and maybe heard Ray and others in the community talking about. And so my focus is really on the verify section of this DevOps lifecycle, right? So after we're kind of done creating, once we've uploaded our code and got it into GitLab, then we're going to verify that that code is ready to go. And so I also, I typically put at the end of my title, parentheses CI, right? So this is all of the CI portions of GitLab for sure, but then also covers all of the testing. We'll talk about that in a little bit. So just kind of how I think about verify is kind of in these three north stars. So one is it's the gateway to operations. So what I mean by that is it's the first stage on the ops side of the house that as we've defined again, the DevOps lifecycle, but an argument can be made for putting it in dev as well, right? So I think that kind of shows how the bridge between development and operations is the verify stage. And we take that responsibility pretty seriously. We want to be able to kind of help bridge that gap for folks that are either new to the concept of DevOps or folks that have been doing DevOps for years and years, but haven't necessarily found the way to really optimize all of that verify step. Speed and scalability is another north star. And so what this is is kind of from the ground up, GitLab CI, even before it was integrated as part of GitLab, leading us to the single application for the entire lifecycle was built for speed. And so it was built on Docker natively and that has enabled speed not only for developers to be able to quickly add tooling and control their builds, but also for just the overall speed of getting things done. And so this is something we think about a lot whenever we're addressing a problem. And then last, GitLab as a product whole talks about lovable solutions, minimal lovable product. That's especially important for us in verify because we're talking about some very complex problems. Again, a demo of CI is great, but our customers and users are really actually solving very hard complex problems and we want to make that easy and lovable. So as you may know, each part of the DevOps lifecycle that GitLab has, we break down into categories. And so these are the categories that we've chosen currently for verify. And so of course, like I said, continuous integration is obvious, that's a big chunk of what we do and a big chunk of where we've built the product to begin with. Code quality and performance testing are things that we've added semi-recently, performance testing in 2018. And then we're trying, for 2019, the vision is to add kind of built-in capabilities for system usability, accessibility, compatibility testing. And so when we say those things, it's an interesting way of dividing testing. The way we think about it is system testing is, not just testing maybe this project or this particular thing that we're running verify for, but the system as a whole. So if you have a microservices architecture, we wanna test the overall system, but that doesn't just apply to microservices. We want it to apply to anything that is a system, a complex system that needs to be tested. So if you think about it as like the project testing is unit testing and the system testing is integrated testing, maybe as an analogy that makes sense. Usability and accessibility go hand-in-hand. Where we're talking about usability, we wanna be able to bring tools to product managers, to designers, to UX teams, to really understand how a change is gonna impact the usability of an application. Accessibility is something that's very important to us as a GitLab. We want GitLab to be accessible in the world. And we also wanna build first-class tools to help everyone make their application successful in the world. And then lastly, compatibility testing can be thought of, again, as a number of different ways depending on the use case, it could be, we wanna make sure that this is compatible with every cloud platform, right? So the Kubernetes group uses GitLab to test against all the various EKS, AKS, Google Cloud, GKE, all of the various Kubernetes installations out there. But this also could be for users that are on embedded hardware and have multiple hardware devices that they have to deploy to. So this compatibility testing is kind of a broad category. We're still looking at what's the minimal viable change that's gonna really help meet this vision in 2019. When we're talking about the community, I think it's really important to point out that Verify in its current form really came from a community contribution. So just kind of like the brief history and I wasn't a GitLabber for this history, so probably I'll get some of it wrong, but DZ, our co-founder and the original writer of GitLab was getting tired of building GitLab as we all do as developers. And so he wrote a little GitLab CI as a separate program and there were a lot of community contributions coming into that, some of the biggest coming from an individual named Camille. And Camille now is a fellow engineer for Verify, right? So he was originally a community contributor, contributed a lot to kind of the vision and what Verify was gonna become. He convinced DZ and CID to integrate GitLab CI into GitLab itself. And so that again was like, that helped set the vision to where we are today of, hey, let's have a single application for the entire DevOps lifecycle instead of having many disparate applications. We originally had GitLab CI as a separate application. And when Camille first said to DZ and said, hey, we should integrate it, they said, no, you're crazy, that's their own strategy and look where we are now. So that's kind of a cool bit of history of Verify that I think is definitely worth pointing out. And then lastly, I just kind of want to talk about some like more recent community contributions, right? I mean, that wasn't that long ago that Camille joined us, but there've been some really cool things done recently and I've kind of categorized them, right? There's lots of things that the community can do to help us like realize this vision for Verify. So it could be as simple as like, this is a one line change, but made a huge difference for people with .NET in our .NET template, we didn't have it set to always upload the artifact. So when the test failed, the test artifact didn't get uploaded, which is kind of silly, right? That's almost when you want the test artifact the most. So we had some of the community to contribute that fix. And then there's also more complex things like complex front end items that really help sorting runners for a user that has a large number of runners, the admin runner page can be massive and hard to understand. And so even just a simple iteration that actually took a lot of work, but as simple once it's done of, hey, let's sort by last contact date on the runners page is really helpful. And then even complex back end items. So this is a really cool one that shipped recently. We've had for a while the ability for you to skip the CI portion when you push up to GitLab by including a little tag in your commit message. But that's kind of sloppy and a lot of folks don't wanna, they might have specific guidelines around their commit messages that don't allow for that. Well, recently GitLab added support for Git 2.0 and that comes with push options. And so a community contributor added a push option for CI.skip that then allows us to keep your commit messages the same, but still skip CI when pushing to GitLab. So that's a really cool addition. Again, seems subtle. It was a big change. I got a screenshot of the MR here with 58 discussions and discussion items and lots of feedback. So that was great. And then honestly, even for those that don't feel like they can contribute that kind of complex code, I think that documentation updates are my favorite. I feel like our documentation serves as great reference documentation, but maybe not the best onboarding and tutorial documentation. And so I've pointed out here that we've got a bunch of merge requests that came in from the community to our docs, but also we have 122 open issues around documentation. If you need somewhere to get started and you don't wanna dive into Ruby or go, that would be a great place to get started. And then lastly, Ray had asked about, hey, what's the one thing you would care about the most, Brendan, if we were doing Hackathon? And so I've got this issue here and it's linked in the Hackathon page as well, which is step folding in CI. So today we've got the log output in CI, which is great and really useful, but it would also be useful to be able to fold those steps so that you can see what are the script steps and then I can fold and unfold the details in the log from each of those steps. This is something that is actually already enabled by the backend. In fact, in this issue, you can see somebody has a little bit of a hack, some like just JavaScript to run on the front end that then folds the steps, similar to that as designed. So I'm hoping that this is like a lower bar for someone that might wanna just get started, but has a huge impact. And again, that issue, if you go to it, has a lot of details, has this design, has details on the CSS and JavaScript that would need to be done to make this a reality. And then I've also got a link here to all of our verify issues that are accepting merge requests, which there's a lot of. So if this issue doesn't strike your fancy, feel free to visit that link. And then also just lastly, I'd say, feel free to ask hard questions. I've got my Twitter and my GitLab handle here. We've got our direction page, of course, that's public like all of our stages. And then this verify hackathon link is just to this presentation. If you wanna get to any of the links in it, you can visit that. Cool, thanks. Thanks, Brendan. Yeah, I was just looking at the issue that we highlighted for the hackathon. It looks like one of the community members volunteered for it, Matthias, and I think one of our core team members helped assign it as well to Matthias. So, but I mean, like you noted, Brendan, there are plenty of other issues, including documentations that people should be able to search through. So there's probably no shortage of things. If people are interested in any of the verify areas that people could tackle, but you're also showing it as well. Yeah, yeah, there's 806 open issues, right? That we were looking at merge requests for. So, like you said, there's no shortage of issues. And then you can filter those even more by these other labels, like continuous integration is a label that would find just the things about CI. This includes, because I'm searching here GitLab org, so it includes things for the runner, which is in Go and things for GitLab, RubyView, all that stuff that's great in the GitLab stack. So you could then narrow it down by project too. Verify is a little not unique, but verify kind of technically splits into the GitLab runner, which, like I said, is in Go and multi-platform. And then all the pieces of GitLab, of course, that contribute to Verify. All right, and I'm sure people have any questions or interested in working on any of these issues that can just mention you on the issue and then they can go from there. Yes, yes, I try to run my life through my GitLab to do's. 36 is actually a decent number, if you can believe it, that I've got right now, but yeah, please just mention Brendan, B-R-E-N-D-A-N, and I'd love to chat about it or help out Harbor again. Cool. Okay, we have a few other participants on the call. I don't know if people have any questions or comments that they wanna raise. Brendan, perhaps a quick question. What would your recommendation be for community members who might have an idea to implement the feature that they would like to see in GitLab, but they might not be sure whether this fits into the direction of GitLab itself. I know sometimes we recommend to find a working in progress issue, but how do you generally manage this on Verify? Sure, that's a great question. I would say, yes, first, like we saw, there's lots of open issues, so I'd say take one stab, one pass through, searching for if that issue exists already, but also don't worry about being the expert on that, that's what I'm here for and we're here for. So if you don't find an issue that kinda matches your idea, just open it, and if you don't ping me, Mark Fletcher who helps us with our issues will, so feel free to ping me. I also monitor, in addition to my to-dos, I monitor the whole Verify label incoming and try to triage that on a regular basis. And then I'll just say that I'm gonna be really transparent, right? I take after our, you know, I'm gonna tell you if I think it doesn't fit in the direction and we can have a whole discussion on that issue and feel free to disagree with me. I am not the only person that understands what Verify needs to be. In fact, I'd probably know less than most, right? Like I think the real-world cases, real-world use cases that we get through issues are kind of the most critical to helping us change the direction and make sure it fits with what users are wanting. So I'd really encourage that. Disagree, I said at the end of the presentation, right? Ask me hard questions. Feel free to disagree with me constantly. My wife will support you in that too. All right, I'm seeing some things on the GitLab community channel on GitHub and attributes related to that. No, I think it's not. I actually have a couple of questions that are more focused perhaps. I've been working with a couple of open source projects who are either in the process of or relating migrating to GitLab and I've had a couple of recurrent questions related to Verify. One of them was having a runner being able to run on ARM64 and the other one was the developer executor for Windows. I'm not sure if they fit into the current direction or if it's something that might go into the category of compatibility testing. Perhaps if you could tell us a few words on these two topics, Brandon? Sure, that would be, yeah. So yes, that fits in with the vision, I think is the most important thing. If you look at our vision page, there's actually a lot of upcoming updates for Windows specifically. That's a focus right now. So today, the runner does run on Windows, but not in the same way, not with the same first class support that exists for Linux, it's just not the same, right? And what that basically falls into today is you have to use a shell runner, which then of course has some limitations in terms of like, that doesn't really fit in the GitLab model of, we love Docker, we love the idea of ephemeral Docker containers for your builds. So the next thing that we're focused on for actually 11.9 and 11.10 is making Windows container executors a first class citizen within the runner. And so you can see there's a lot of issues. If you go to the vision page and the Windows container executor, there's actually an Epic that has what's included in the MVC and what's included, what's not, I can walk you through real quick. So I'm just on the verify page. I know that Windows container executor is listed for 11.9. And if I go to that, we should have the epics that are related. I'm just letting it load for me here. Yeah, so this is part of the MVC container, Windows container epic. And so we talked about what's in and what's out of kind of the epic and where those things are shipping. So we've actually already shipped some pieces of that. The biggest hurdle we're having right now, not hurdle, but the thing that's taking the longest right now is just shipping helper images. So we have a helper image that downloads next to your image. And we have to have an individual one of those for each version of Windows because of the way Windows containers work. So that's just a heavier lift than we originally expected. But we're really focused on this. It's kind of the number one thing for the runner team right now. Arm64, I don't know off the top of my head. I thought we had support for it. So if we don't, we should. And I'm gonna look into that for sure. Because again, the point of runner being in go theoretically and in reality is we want it to be able to run everywhere. So like down to the point where we've got users and customers that are interested in running it on ZOS from IBM name frames. And we're working with IBM's engineering team on how that can work because ZOS supports go now. So I would say Arm64, if we don't support it today, I'm gonna go look after this and I definitely want to support it. There's an existing much, much request and I'll pin you on it because I think when first service discussion when I first got involved, you weren't yet managing verify product. So I'll make sure that there's been, there have been due on that. Essentially there's a medium that created a branch. And he's been, I mean, essentially, there is a way to do it but this is how we work around this right now. Gotcha. Yeah. I'd like to make that not, you know figure out what the engineering team and how we can make that work also without workarounds for sure. And then, I mean, I think just generally, I mean, that's platform support but on the executor support side, there is, I think, I don't know if it's an existing contribution or one we're working with folks on the idea of a generic executor. I think that would be really great for the runner because I think a lot of ideas are like, hey, we want this kind of executor, hey, we want that kind of executor. And if we accept every executor coming in the door, we then have to accept support for every executor. And so I would like to see a generic executor that we can support and then the community can extend if they've got a bespoke system that we don't have in our testing stack and have the ability to support directly today. And so that's another thing that you could look for on the community side. Awesome, thanks so much, Brandon. All right, any other questions or comments? We have about a few minutes left, but if not, I'm happy to end this early. Brandon, thank you for your time, not only for leading the session but putting the materials together and also getting the links out too. So we'll get this posted on our YouTube playlist and hopefully in the next couple of hours. Awesome, thanks, thanks for putting this together. Really helpful. Thanks, thanks, have a good day. Thanks everyone, take care. Bye.