 All right. Hello, everybody. Welcome to the UX at GitLab webcast, a deep dive into GitLab's UX process. Very quickly, I'm just going to go over the speakers that you'll be hearing from today. First is myself. I'm Sarah Vesloff. I'm the UX manager here at GitLab. Catherine Okapara is our junior UX researcher, and Tori Davis is our staff UX designer. Dmitri Hookstra, who won't be speaking, will be helping out by sharing links along the way. So as we talk about specific resources or issues or things that we'd like you to visit, he'll go ahead and put those in the chat. And feel free to post your questions using the Q&A feature. We're going to answer them all at the end, but you can post them anytime. You should see that little Q&A feature on the menu bar. So if you're not familiar with GitLab, I'm going to tell you a little bit about us first before we dive into the UX team itself. At last count, the entire GitLab team, at last count being yesterday, I think, the GitLab team consists of 339 team members and there's 78 pets. We're a remote only organization, and we currently have team members in 42 countries and regions. So we have no home office, there's no central location, there are not offices in different countries. We all work remotely from our homes or co-working spaces or wherever is convenient for us. As an open source company, we do everything in the open. And just like our product, our team handbook is open source and 100% public. So anyone in the company or outside for that matter can suggest changes, additions, and make edits to it. So that includes you. If you see something, you see a typo, or you think something could be worded better, go ahead and make a suggestion. And most likely we'll make a change. And you should see a link to the handbook in the chat. So the handbook also includes an entire section devoted to the UX department. We're going to go over a bit of that here today. But if you're interested in some full details or maybe there's something we don't have time to answer at the end, I suggest you peruse through the UX section of the handbook. It has a lot of really good juicy details on our day-to-day operations and how we think about the work we do here. As of right now, the UX department is made up of two UX researchers and 10 UX designers. Together we expand eight countries and six time zones. We are hiring. That's why I say as of right now. So please stay tuned at the end of the broadcast to learn more about these open roles and be sure to post any questions you have about those roles. So if you're curious about what it is we're looking for or what kind of skills you need, we're more than happy to elaborate on that at the end. So how we work. I wanted to make a quick note here. You're going to hear me use the term UXer during this webcast. We kind of went back and forth on whether to use it. It's how we refer to members of the UX department, both researchers and designers here at GitLab. So when you hear me say UXer, that's anybody that is on the UX team. Just out of habit rather than saying designer or UX designer. It's kind of our little short code. So just an FYI. And I do want to touch just a little bit about on GitLab itself. I think it's important for context to understand what GitLab is and what we do if you're not familiar. And that will help you understand a little bit more about the unique way that our team works. So GitLab is a single application for the complete DevOps lifecycle. It's the first single application built from the ground up for all stages of the DevOps lifecycle. It enables the team to collaborate and work from a single conversation instead of managing multiple threads across disparate tools. It provides the team a single data source, one user interface and one permission model across the entire DevOps lifecycle. This allows teams to collaborate and significantly reduce the amount of time that they need to work together. And as you can imagine, given what we do, our user base is broad. It consists of developers, project managers, UX designers, product owners, office owners, and business owners. And it allows the team to work together and work together. And it allows the team to work together and work together and work together. And it allows the team to work together. And it allows developers, project managers, UX designers, product owners, operations management, and many more that I don't have time to list. There's a lot of different people that use our product. And here at GitLab, UX is actually part of the engineering team. And that means that each team stage has a dedicated UX designer. Oops, I started a little bit late. Let me back up a little bit. We do have two departments here at GitLab. We have Dev and Ops. And within these two departments, we break things down further into team stages. So each of those team stages will have a dedicated engineering team and product manager. And since UX is part of engineering, that means each stage has a dedicated designer. So the UXer on each team is responsible for the same features that their PM oversees. The UXer will work alongside the PM at each stage of the process. That means planning, discovery, implementation, and further iteration. And the area a UXer is responsible for is part of their title. So you'll see UX designer plan, UX designer release. And you can see each of these aligned in our team org chart. And you should have a link there to go see our team chart. You can filter by UX and you'll be able to see where every person works. UXers can also serve as a backup designer for other areas of the product. The area will be listed on the team org chart under their title as kind of an expertise. So you may be the UX designer for plan, but you're also a release expert. And that means that you can kind of pinch hit if someone is unavailable or we have an unusual amount of issues in one area at a time. And we follow a monthly release cycle. So we ship on the 22nd of each month here at GitLab. We plan and track these monthly releases using milestones. Our release timelines are overlapping. So for example, when a release is shipped to production on the 22nd, the scope for the following release has already been established earlier in that same month. This visualization here kind of gives you an idea. It's one of the hardest things that I had to learn here at GitLab when I was starting. I had a really hard time with what milestone are we in? Because you're really wrapping up one as you begin another. So each month the team, meaning the UXer on the team, the product manager, the front-end and back-end engineers will communicate in the issues about scheduling and prioritization. So they'll have conversations back and forth about what we know, what we don't know, and whether this can be done in one release cycle. Once the issues in a milestone have been set, the UXer will assign themselves to the issues in the area that need UX attention. And they'll try to gauge whether they have time for all of the issues, or they have to go ahead and push some off to the next release. And as a 100% remote company, we do work asynchronously. So we use GitLab issues to propose and discuss UX improvements across the product. All of our discussions happen within the issues. This includes adding notes from Slack conversations or the occasional asynchronous or synchronous meeting. So we'll go ahead and make sure that we're linking to a document from a meeting or linking to a Slack thread so that everyone is involved on the conversation and no one is missing out. Collaboration is one of our core values here at GitLab. It's about helping each other when we have questions, need critique, or need help. It's not about brainstorming or waiting for a consensus. As we work through individual issues in a milestone, we involve the entire team in the discovery and design process. That means engaging engineers, product managers, community members, customers, and other UX designers on the issue for critiques and input. However, we do make the final decision. So again, it's not a design by committee, which can be disastrous as we all know. It really is about collaboration and getting all the information you need before you make that decision. So the UX department has a shared project on GitLab. This project houses all of our work in progress and finalized files along with prototypes. So this shared repository helps ensure that we're able to find the files and designs whenever necessary without needing to reach out to another member of the department. They're always there in the repository. Each issue is different, but we'll often begin with designs and sketch. You will see pictures of sketches. So we'll take images of sketches maybe that we drew by hand or wireframes from other programs or even wireframes that we made in Slack. Whatever makes the most sense for the issue we're working on. And we share those designs in the issue, asking for feedback and iterating on it until we get it right. Once it's complete, that design is implemented inside of GitLab. So we've talked a lot about how each UXer is dedicated to a product stage, but they're still part of the larger UX department. We meet together on a weekly basis to discuss announcements, initiatives, design ideas, process improvements, pretty much anything we think is important. We'll share articles, we'll share redesigns that we think are really cool. It's kind of our opportunity to socialize and talk shop. This helps us stay aligned on our vision and goals for UX at GitLab. And these calls are recorded in attendance is not mandatory. Important to note as an async company, we try to accommodate everyone's different time zones and schedules and family needs. So as a manager, I will meet each member of the department in a bi-weekly one-on-one. It gives us the opportunity to talk about challenges, opportunities for career growth or personal growth here at GitLab and beyond. And finally, the most exciting, of course, the entire company meets about every nine months somewhere in the world for the GitLab summit. The summit allows us to get FaceTime, build community, and get some work done. Our next summit will take place next week in Cape Town, South Africa. And there should be a link for you to check out the details there. So next I want to touch on iteration. It's an incredibly important value here at GitLab, particularly for the UX department. So I'm sure that you've heard the term iteration. It's very common, common in design, common in agile methodologies. Here at GitLab, iteration means making the smallest thing possible and getting it out as quickly as possible. So small and so quickly that it sometimes is painful. We know it's painful. You might feel embarrassed by how unfinished or unpolished your suggestion seems, and that's okay. If you're doing iteration correctly here at GitLab, it should be painful. It should hurt just a little bit. Working like this is extremely important for us. It allows us to reduce the cycle time and get feedback from users faster so we can continue to improve quickly and efficiently. And iteration isn't just one of GitLab's six founding credit values. It's one of the foundational concepts in design, thinking and user experience. Planning too far ahead without getting real-world feedback can cause you to build something that just doesn't meet user needs. It wastes valuable time and it can bring the team so far off track it can be hard to recover. And there should be a link here to our values. It's an interesting read if you have some time. And we really think that iteration is especially vital in an open-source community. Keeping changes small and iterative makes it easy for anyone at GitLab or in the community to contribute, and that means you. We keep a list of improvement issues that are actively seeking contributions from the community. They're small in scope and they allow the community to contribute designs or code to issues. And I'm going to talk a lot more about this later on and give you an idea of how you can contribute here at the end of the presentation. So I'm going to stop talking for a little while and give Catherine a chance to talk about how we do UX research here at GitLab. Take it away Catherine. Thanks Sarah. Okay so let's start with the general overview. The goal of UX research is to understand the needs and concerns of users often by observing how they interact with the product and other times by gathering data through methods such as surveys and user interviews. Our UX team specifically strives to meet the needs of our users while balancing product requirements and milestones. Now this can often be difficult when you need to juggle a lot of different opinions and expectations. So having user research to back up design decisions will keep you from second guessing yourself when you receive feedback. Next slide. If someone is interested in exploring your research question, we encourage them to create a new issue using the search proposal template in our UX research project. This template encourages contributors to describe their questions and identify any assumptions or theories they may have. After reviewing proposals, we determine the types of studies to conduct. Some of our most common UX research methods include survey research, usability testing, user interviews and card sorting. A great way to choose research method is to ask yourself what you'd like to learn, who you need to reach, and how you want to communicate your questions to the participant. For example, that could be visually, verbally or both. Next slide. After deciding on a method, we recruit participants from our research panel and conduct a study. When a study has been completed, it is important to analyze the data and interpret the results in an unbiased way. It's often helpful to outline the factors, if any, that could have influenced the results and discuss other possible interpretations. This helps us keep a balanced view of the situation and prevents us from confirming only our own hypotheses and assumptions. At the end of each study, we create a report and communicate these results to everyone involved in the issue. From that point on, we use the results as a way to identify actionable steps forward. Discussing the results of our research with other designers and product managers helps us prioritize feedback and determine the next steps to take to implement the findings. UX research is an important aspect of product design that can occur in different phases, whether exploratory or evaluative. UX research is a valuable way to gain a deeper understanding of the users. Next slide. GitLab's navigation redesign is a great example of the product area that has benefited from research and design iteration. From March to April 2017, we conducted three usability tests to observe how users navigate throughout GitLab and to understand where they encountered issues. These tests were moderated, which means that a researcher was present as the user completed different tasks. The initial test highlighted a large amount of usability issues in various aspects of our UI, such as the project navigation, sidebar, and search bar. This gave our designers a detailed report on problem areas and open issues that needed to be addressed. The follow-up usability tests were conducted as a way to test how much progress had been made after improvements were implemented. These tests showed us which issues still remained and which issues were successfully resolved. Next slide, please. After highlighting a large amount of usability issues related to contextual navigation, we then conducted card sorting tests. Card sorting helps you discover how people understand and categorize information. The goal of these tests was to determine the optimal structure of the contextual navigation at a project and group level. 72 users participated in an open card sort of project level items, while 46 users participated in an open card sort of group level items. Users were asked to organize navigation-related content by creating categories and organizing the content in those categories. After analyzing the results, we created a recommended contextual navigation structure based on the way that the majority of users categorized the content. Next slide, please. We conducted a round of tree testing after the card sorts to test how well users could find the information they needed when using GitLab's existing contextual navigation versus the optimal one we highlighted in the card sort. Tree testing is particularly useful for tracking and mapping the different paths users take when they are searching for information. We used the results of this study to identify five key changes that could be made to the project and group level contextual navigation. This study helped us organize the menu items in a more intuitive way, and it also just gave us general insight on the ways that users interpret the names of items we include. Next slide, please. As a result of these continuous efforts to improve our navigation, users who were once frustrated by their experience navigating GitLab became supporters and contributors to the effort. As we move forward with our user research, we now have qualitative data to measure future improvements against. A key part of our practice moving forward will be gathering baseline data on features before introducing changes. Next slide, please. We have created an archive in the GitLab handbook to serve as a place for all product contributors to learn about the reasoning behind design decisions, and also to see a history of the progress we've made. The UX Research Archive is a collection of all the reports from all the studies we've ever conducted. Feel free to take a look through it to learn more about UX Research at GitLab. Next slide, please. The research panel is our database of both GitLab users and non-users. We send out invitations to participate in surveys, interviews, card sorts, and more. If you want to get involved in our research, please join the GitLab Research Panel using the link in the chat. Next slide, please. And now I'd like to introduce our next presenter, Tori Davis. Tori is a staff UX designer at GitLab and a design enthusiast that years of experience. On to you, Tori. Thanks, Katherine. So one of our major initiatives that we started last year was our design system. And for those who are unfamiliar, a design system has multiple different parts that often include content guidelines, usability patterns, foundational styles, and reusable components. And these are a lot of things. We knew that implementing a design system into GitLab, an already pretty complex application, would be challenging. And we thought that even though it would be a lot of work, the benefits definitely outweighed the time investment that it would take. Next slide, please. So to start, we identified some of the primary benefits that a design system would give us. We knew that by shifting our focus towards system thinking, we would be able to create consistency throughout the product as well as predictability among our experiences. We also knew that designers, engineers, and product managers would spend less time reinventing the same solutions, and that would give us more time and energy to focus on the problem that they were trying to solve. And with a set of reusable components, we would also simplify our code base and create more autonomy among the teams because the clear documentation that we were writing would give everyone the ability to reference and use without having to ask someone on the UX team for clarification. Next slide, please. So I mentioned before that a design system can simplify your code base, and we knew even before starting this process that we had a lot of duplicate code on the front-end side. There's also a lot of minor inconsistencies that you can see throughout the products that we're completely aware of before we even started this process. We did an inventory of the application and found that we had 82 different variations just of the color gray. We also had over 32 different typography styles, which actually isn't that many if you take into account all the different states like hover and active states, but they're also defined in percentages, rims, and pixels, so there wasn't really a cohesive or predictable type system in place. Next slide, please. So Pedro, who is also a designer on the UX team, worked on an interface inventory, which documented all of these inconsistencies that we were finding, and many of these were issues that we were all aware of, like I mentioned, but it was nice to kind of get them all in one place to visually see. If you've never done an interface inventory before, it's pretty easy, although a little monotonous as well. You just go page by page and take screenshots of all the variations of a certain component and then paste them somewhere so you can easily reference them later. Next slide, please. So from there, we were able to start looking at what made up the foundation of our designs, and this included a solid typography system along with a structured measurement rules and color guidelines. And for us, building this foundation first was really important because it was the base that allowed us to build the rest of our components. Next slide, please. And we didn't nail it right away at all. It took us some discovery to really figure out what worked best for our product. Typography was a huge one and we ended up creating a series of three typography systems to use, depending on the application. We also introduced an outlier within our measurements that you can see in this slide, which gave us the ability to have better horizontal spacing for things like forms and buttons. So in this example, you see on the left, we started with 16 pixels, but it was way too wide and then eight was too narrow. So we ended up going with 12 and that was our outlier number. So even though we started with these base styles, we made sure to apply them to some of our components and make sure that they would work well. And then if they didn't, we just adjusted them along the way. Next slide. So a large part of the shift was to start thinking about design systematically and in order to do that, we use Brad Frost atomic design principles, which is fairly well known and common now. This approach breaks components down into atoms, molecules and organisms. Atoms are essentially the smallest piece of your product. So a button, icons, things like that. And when combined together, you get molecules. And when you combine molecules together, you get organisms. So here in this slide, you can see an example of the molecules that make up our contextual sidebar. And the modifiers you see are the different states that a molecule can have, like a hover or an active state. So by using this system, we were able to start building out the most commonly used atoms in order to create, then create the molecules and organisms. And we did this within sketch. So now we have a pattern library that we can reference as we build out our documentation and actually implement the new system. Next slide. So a design system is more than just a pattern library within your design tool, right? To get all of the advantages, we knew we would need to include these three parts, usage guidelines, front-end guidelines and live code examples. And one of the key differences between a design system and a pattern library are those live code examples. We don't include images in the design system because they can easily get out of date and are difficult to maintain. So instead, we're really focusing on creating a system that has these live coded examples, which will also be used in our products. So they're always in sync with each other. Next slide. So we've already started writing usage guidelines for multiple of the areas of the design system, but there's still a lot to do. We have a lot of work ahead of us still. We're working really closely with our front-end team to implement our system iteratively. And if you look at the system right now, you'll see there are a lot of to-dos. And that's because at GitLab, we don't really believe in keeping everything hidden until it's perfect. We have everything out in the open, and we're constantly making changes and improvements as we go. Next slide. And right now, we don't have a dedicated design system team. Every designer on the team is writing usage guidelines every milestone. So we each pick at least one issue within our issue tracker to contribute to the project. And our front-end team is working on laying the foundation that will allow us to implement the updated components throughout the product. So even after all the to-dos are filled in, we know that the design system will never be finished. And it's open source just like the rest of GitLab. So everyone is encouraged to question any of the decisions we've made or contribute by making things more clear or adding missing content or any other way. And now I'll pass it back off to Sarah. Awesome. Thanks, Tori. Really appreciate it. So now we're going to talk about how you can contribute to GitLab. Probably the most exciting part for me. So again, as an open source company, we believe in transparency. We share almost everything we do. Anything that we can share is shared. Source files, artifacts, deliverables, case studies, UX research, and our findings. And you might be asking, why? Why would you share so much of the work that you do? And it's because we believe that we can do more together. Being open source allows the community to learn from us and for us to learn from the community. So why should you contribute? I think there are a lot of reasons. But I pinpointed a few here. First, contributing to GitLab gives you a chance to collaborate in an open community and to work with passionate people from different parts of the world. It's an unusual opportunity. I can say that this is the first time I've ever worked in a company like this. And it has been inspiring. That sounds really cheesy, but I mean it. I'm just astounded every day. I'm challenged every day by the experiences and the different viewpoints of the people that I work with here. And I think it's something that if you have the time to dig in and be a part of, it's really worth it. And another thing, if you're just starting your journey as a UX designer, you'll have the opportunity to work alongside the entire UX department here as well as engineers within the team to solve real-world problems. I think one of the biggest things that I hear when I go, when I do mentoring and I speak to those coming out of boot camps or just finishing up their degree is how do I get experience? I have this degree or I have the certificate and I have the drive and the passion, but I can't get a job until I have the experience. And I can't get the experience to get the job. And so this is a really fun way on your terms to contribute what you're interested in, work on things that are interesting to you and gain some experience that you can take with you into the job market. And if you're a seasoned UX designer, you'll have the opportunity to explore new problem spaces and mentor junior designers. Maybe where you are right now, you kind of feel stuck, stagnant. I've been there before where it's like, ah, I know how to do this. I've been doing this a long time. Maybe even I could do it in my sleep, you know, and you want that challenge. You want something that makes you sweat a little. And let me tell you, some of the problems we face here at GitLab will make you sweat. They're really interesting and complex and require a lot of thinking. So it can be really challenging and interesting to help work on this product. Anyway, not be sure where to start. I think it can be overwhelming, particularly when you look at the DevOps lifecycle and all of the things that we do that can seem complicated and intimidating. They're not and we're here to help you. So we're going to link in the chat to a list of issues that you could start with that I think are a nice ramp up. They're issues that have been labeled accepting merge requests and they need some UX work. So accepting merge requests is a label that we give to issues that have an idea we really would like to implement in GitLab, but we just can't prioritize at the moment. So we'll mark it with the accepting merge request label to let the community know that we would love for someone to pick it up. And when you see the UX label, it means that it also needs the attention of the UX designer. So in some way, shape or form, it has an effect on user experience. We want to make sure that we think about that before someone goes and actually does a merge request to add the code for this. Most of these are very small issues and it's the perfect starting point if this is your first time contributing. But you may also have an idea for your own UX improvement. Maybe you use GitLab at work or you've used it casually for some projects that you've been working on and there's something that just drives you crazy or really bugs you. We welcome your suggestions. In fact, I can't tell you how much I look forward to issues that are created by our customers and by our community. They're important because you're the user. This is something that you're experiencing in your workflow that's slowing you down and making it difficult for you to get your job done. So to suggest a new feature or user experience improvement, we actually opened a new issue in our Community Edition project. There should be a link in here for you to check it out. So you want to start before just jumping in and opening an issue by doing a quick search and just see if anyone else has made the same suggestion. It happens a lot. So we try to make sure we're not making duplicates. But it does happen, so no worries if it's a duplicate. We can always merge them together. But once you're fairly certain it isn't, if you weren't able to find anything else referencing the problem that you're having, you can create an issue using the feature proposal template. Just fill in each section of the template, describing the problem you're trying to solve and your proposed solution. And you want to make sure to follow the contributing guidelines outlined so that you can add the correct labels. So make sure to check out those contributing guidelines. Who to pay? You can reach out to the UX department for help and guidance on any of this. Simply use the at-gitlab.com or gitlab-com forward slash gitlab-ux. Not exactly simple to remember, but it's there on the slide. If you insert that into a comment or in an issue, it automatically pings the entire UX team and lets us know, hey, someone needs your assistance. Someone has a question for you. So use that to bring attention to your issue or bring attention to something that we're interested in working on. And we'll try to get back to you as soon as we can. And finally, we do currently have three open positions for UX designers. We're looking for UX designers with an interest in our product. So whether that's the development side or the ops side, if it's a code review, portfolio management, continuous integration, Kubernetes, we have a lot going on and we have something really to offer for anyone. So if you're interested in checking out our openings and seeing what it takes to get on board here at Gitlab, please take a look at the links. There should be a link to the description for a UX designer here. And then there's also a link to apply. The slide shows the open teams. So we're actually recruiting for specific teams that are going to need a lot of UX work. So right now that is our release and verify, monitor and secure teams. And I just want to thank everybody who has come to listen to us, talk about UX here at Gitlab. It is definitely something we really enjoy talking about. And I really appreciate you taking the time out today to listen in and hear about what we have to say. It looks like there's some Q&A. I'm going to go through and answer some of these. And I'm not sure how to do this Q&A. So let's go ahead and do this together. It looks like some of the questions people want to answer live. So does somebody want to jump in, Tori or Dimitri and open these up and answer? I think it might be smart to begin with the first one. The oldest one has been asked and go all the way back. The first question would be, what do you currently feel is the biggest UX related challenges at Gitlab? Would you answer, Sarah? I knew you were going to kick it back to me. Thanks, Dimitri. Yeah, it's a really good question. So I think one of the biggest challenges that we have right now is growing the entire department at the same time that the company is growing and trying to make sure that we're able to address all of our plans for the vision. So there's a lot of features that we have planned and there's a lot of improvements we want to make to existing features. So you kind of have to do this balancing act between adding value to the product and making sure that we're doing everything we can to be useful. At the same time looking at issues or not issues, but features that are already out there that need additional improvements. So that can be a really challenging thing, right? It's like, do you do this or do you improve this? Which is going to give you more bang for your buck, more for your time spent? So we do a lot of that. And I think that's a lot of the challenge that we face as individual designers on the team, kind of that push and pull between improving certain things and then adding additional features. I'll just go down and read some of the questions. Tmitri, Tori, Catherine, if any of you want to jump in and answer any of these, please let me know. So the second question, you mentioned sharing the sketch in the issue in the beginning and the iterating based on feedback. Do you get customer user feedback at this point? Yes. We get a lot of feedback from customers, from community members, as well as from other engineers and designers on the team. So it's really interesting. So you'll have this conversation in the issue. So it starts with a description like this is the problem we're trying to solve. This is how we think we might be able to solve it. Go. And then you start talking back and forth of maybe I don't think this is the right solution or have you thought of this? And at some point it gets to the stage where you actually want to sketch it out. And once we do that, we'll put that in a comment and say, what do you think of this? Does this solve the problem? And that's when people will chime back in. And often help drill down to the actual solution. A lot of times we find that over time, our solution gets smaller and smaller, which is actually better, right? We're stripping away all the things that we don't need to do and we're concentrating on just the thing that's going to solve that problem. So here we go. When you put out something very crude early iteration, do you roll it out for everyone? Or how do you roll it out usually? Quite often we just roll it out. Again, iteration is painful. Sometimes you put something out and you think, oh, jeez. Oh, I'd like to do this and I'd like to do that. And I wish that I had time to do this, but you have to resist that temptation. It's really important to get it out there early and make sure that you're on the right path. So often there are things that will roll out as long as it is an improvement over the existing user experience, we will roll it out. If it's something that could have a high impact, so there's a lot of risk to rolling it out, we will be more careful. So the navigation redesign, for instance, there's a lot more risk in that, right? People use the navigation every day to get from one area of the product to another. And hindering them is dangerous. So that we did roll out in a beta and allowed people to opt in. So progressive disclosure, right? People could opt in. We created a feedback issue for our customers and the community and other Git labbers to say, hey, this is working or this isn't. And it allowed us in two releases to completely redesign the navigation and roll it out for everyone. We're also utilizing feature flags for some features that may be more risky. So we implement a feature flag and we can roll it out to a specific portion of Git lab, but we do try to make sure it's an improvement that everyone will not feel like is risky to them. So we don't have to use feature flags that often, but it is an option for us. Thanks, Tori. So the next question. For people starting out in UX, what's your main recommendation to get skills on? This is a good question. There's so much. It could feel overwhelming, right? I think that for UX, getting experience on several fronts is really important. So one is the basics of design thinking and design skills. So understanding how as a designer, you're going to approach a problem, think through a problem, and try to get results and solutions. How are you going to visualize that problem? So wireframing, typography, color theory, layout, those kinds of foundational elements of design are extremely helpful. I see a lot of people come from other fields and they have transferable skills that they can use. So experience in research, whether it's through a degree that you were pursuing that included research, or just something that you enjoy doing for yourself, kind of researching different topics, that's a big help in UX because so much of what we do is about asking questions and analyzing the answers that we get. I also think that if you intend to work on, in the technology field, it is important to at least have a basic understanding of development. That doesn't mean that you have to be a developer. I'm not saying in any way that to be a UX designer, you have to also develop. But it's important to understand how your designs are going to be implemented. So even if you're not working for a tool like GitLab, which is for developers and for people that are working in code every day, it has to be implemented in code, right? Somebody's going to have to build it in HTML, or CSS, or USAS, or maybe they're building in React and they have to componentize everything. Understanding that as a UX designer is really helpful when you're putting things together. It's going to make that handoff and that conversation a lot easier. And it looks like the next question is good for you, Catherine, maybe the next two. So the next one is, how do you combine analytical data in UX research? Sure. A good example of that would be, say if you have a problem area in your site, like a certain feature isn't being used a lot, you could run analytics on that feature and try to get the baseline data for how many people visited this feature and has there been any big change? Like, did we just make an improvement to it or basically a change that might have impacted someone's opinion on it? So you can get a baseline data and then run some research to test, okay, how did people react to this change? What did they like about it? What did they not like about it? So you can mix that together as a way to kind of balance the analytical approach with more qualitative approach. That's what I'd say. That's a good example of that. And for the next one, how did you recruit users for card sorting, tree testing, and et cetera, and do you compensate them? So yeah, so we maintain a UX research panel. Basically, we have a sign up link on our, on our marketing website where people can opt in to receive invitations about different studies that we send out. And every time we have a study, we basically ping a list on MailChimp and say, hey, if you want to participate, you have a chance to enter a prize draw for Amazon gift cards and whatnot. And so that's usually how we handle it because we receive a lot of responses. So it'd be difficult to pay every single person in the case of a survey, for example. So we do a prize draw. If it's a smaller group of participants for something like user interviews or usability testing, then we'll pay each participant. That's usually how it goes. And next question. You mentioned wanting to gather baseline data before introducing changes. How are you thinking to approach this? I think I'll answer this as well. We're looking into different analytics solutions that we can use for different features on JetLab. It's a little challenging because we need open source options. So we're still trying to figure that out, basically. But basically, the best approach to that would be actually installing some kind of analytics tool that's gathering data in the background so that we can check and see how often people are using certain features and kind of when the different changes happen. So did usage of a certain feature change after a certain release? Why or why not? What was different around that? What was different in the context we could use to make those plans positioned? Yeah. And on to the next question. Yeah, I can. So I'm going to pass this off to the next question to Clement, who is here from the front-end team. It's regarding our internal structure and thoughts based off of the View One Year Later blog post. Just our choices made with View, View X, and maybe other approaches related to that structure. That might have changed or improved since then. But do you mind answering that one for this person? Yeah, I can answer. I'm not 100% sure what the question is, because I haven't seen that blog post in a while. But basically, with the design system, the front-end team is also trying to set aside time to build components that implement the design system as well as CSS framework that influence the design system so that both code bases for Hamel, non-View code can still use our styles, as well as View code can use our styles. And as we create more deliverables, create more front-end work, there's a higher degree of view stuff that's being written. So there's still a possible chance that one day maybe GitLab will be an all-view application, and then all our design system stuff will be view components. But for right now, since we still have a lot of things in Hamel, a lot of our design system will be more CSS-based first. But we also have these components that automatically have the styles. So we kind of have this dual approach for now. But yeah, that's kind of where we're heading at this point. Thanks. And next question, I think, is back to you, Catherine, about our research panel and sourcing. Right. And I'll pass this one on to Sarah O'Donnell, who is a senior UX researcher at GitLab. So this question is in relation to how we manage the research panel at GitLab. So what we currently do is actually we recruit users through a MailChimp link, and then the details of the users are stored within MailChimp. We segment the list within MailChimp, so we don't send too many campaigns to users and bombard them. We want to keep active users on the research panel as well. So we're looking at how many studies they kind of take part in and how many they've completed recently, because we don't want any kind of bias. And we don't want to kind of hear the same voices over and over again. And again, and then we just kind of offer incentives ourselves. We pay normally through Amazon gift card. There's a lot of manual management of a research panel. I'm yet to find a good solution out there. So if anyone has any recommendations then please do let me know. It is a very manual process, but it seems to work for us so far. Great. And next question is about how we keep and maintain our library and sync with the current production code. And this is something that we are working on right now. Essentially the code will be synced between the design system and the product. So it'll just automatically be in sync all the time because it's using the same code. I don't know, Clement, you want to add anything there or not, but essentially it's just using the same code. So it'll always be in sync. I think, I think a side note to this question would be that perhaps this can be seen as like, how do you keep the pattern library in sync with the production code, right? And sorry, could you perhaps elaborate the question from, from that perspective? Yeah, so the pattern library, we, if we make a change then the change is coming from the pattern library typically. And where we want to get to eventually once the design system is in place is that we don't have to worry about the sync between the pattern library and the design system because we're all working from the design system. So it's not about like keeping everything and sketch completely 100% what production is because we have the design system in place and we don't need to do that anymore. The pattern library is really a step to the design system. And then once we're there, that's what we'll be working from. And it's really a chance for designers to kind of make prototypes in the browser more easily and things like that. So we won't be relying on sketch so much as we are right now. Hopefully that answers that question. Thanks Dimitri for pointing that out. I didn't realize that's what I was asking about. And then Sarah, you want to take the next one. Sure. So the next one is, is there a recommended timeline that you follow for each iteration to the product? I'm going to say no. I think anyone else can chime in here. You know, there isn't a prescribed timeline. It's not as though we have an idea for something. And then we decide that we're going to iterate this and this milestone and then this and then this and then this. Really, we do approach it milestone to milestone. So we try to make the smallest change possible, something that's doable for the team inside of that milestone and get it out there. And then in the next milestone, if it's still priority to continue iterating on that, we will. Sometimes it may not be. Sometimes the goal is to get it out there and then gather feedback and see what do people think of this feature. Are they using the feature? So that allows us to go and work on other iterations of other features or new features. And so in that way, I wouldn't say that there's a recommended timeline. Really, the baseline is, are you making the smallest change possible? Are you making sure that you're getting things out in a quick timeline so that you can gather that feedback quickly? And that's kind of our barometer for how we're iterating. And the next one is really interesting. I think it's a really good question. So the next question is, how do you tackle the somewhat primal need, I love that word, of being physically present when discussing some topics, such as brainstorming, co-planning, or prioritizing? How do you actually collaborate during a sprint cycle? Oops, I think jumped. There we go. Does it change if it is a microtopic versus a macrotopic? I'm going to answer a little bit on this. And then I'd love it if Tori or Dimitri or Sarah or Katherine wanted to jump in here. I think that there is this urge sometimes to have a meeting, right? You're talking back and forth in an issue. Maybe you're not sure if you're completely understanding what the person is saying. And you kind of have this go to of, well, we'll just jump on a meeting and talk about it. And that'll clear it all up. The problem with that is that it's just you and that other person, most likely. Other people are not privy to this conversation. They're not going to hear the what, the why. Even if you get a summary put into the issue, they could miss a lot of the context. And it's just not efficient, right? Having to wait for people to be able to get together and actually be in the same space just isn't productive. It slows us down. So all of our collaboration is really done within the issues. Do we have meetings? Absolutely. Sometimes there'll be a kickoff meeting, which really isn't about brainstorming so much as the planning and the prioritizing. But we try to minimize that as much as possible. Tori, Dmitri, anybody else want to jump in here? No, I think that was great. We only have seven more minutes and a lot of questions. So we can just keep going through, I think. Okay. All right. Oh, a great question about research. So Sarah, I think the other Sarah, not me, obviously, you want to jump in here. Sure, Sarah, could you tell me what the question is? Oh, so this one was about design focus issues that people can contribute to. So I don't know if you have issues that you can link for them to contribute to, Sarah. Oh, for research. Yeah, the question is these don't really seem research focused. What if you want to help with research related problems? Is that something that we're going to put together? Do you think Sarah, do we have maybe some time to gather some stuff that people could assist with? Yeah, if people want to be involved with research, by all means, we are always interested in people who can help us with things like competitor analysis. We are even happy to do some usability testing if they're willing to kind of work alongside a researcher at GitLab. So we can definitely put something together if people are really willing to be involved in research. That'd be fantastic. Awesome. And the next question, sorry about that. No, you go. And the next question is how would you describe the UX research demand versus output capabilities? And how does it tie into the current amount of UX researchers? Everybody wants research at GitLab. This is why we don't bother an extra pair of hands. People who would like to help contribute towards the research efforts at GitLab. We would love to speak to you. You can reach out to me directly at GitLab. My pin name is at Sarah Odi. And tell me what kind of research you've done and what you'd like to be involved with. That's awesome. So the next one is what is something you wish you knew when you were starting out as a UX designer that you know now? I'm going to kick this to Tori. Okay. Something that I wish I knew when I started out as a UX designer that I know now. I think that I, the focus is on problem solving. So something that I may not have fully realized when I was first starting as a UX designer is that it's not really about the solution. It's about the problem. So as long as you are actively trying to figure out what, what is the real problem here that you're trying to solve, I think you'll be in a much better position to ask the right questions in order to find the right solution. Let's see. And then the next question is as UXers in the engineering part of the company, did you have to create the live coded examples instead of sketch slash static images? We're working with our front end team to do that. So this also relates to the next question about who's coding the live examples. The bulk of that work I would say is on Clement and his team. So they're working on the actual JavaScript components and they're working on that. And then on the designer side, we do, we're hiring for designers who have some development background so that they can contribute in terms of like HTML and CSS and JavaScript is capable. So it's kind of a joint effort, but mainly the front end engineers are working on those live coded examples. And how long does it take commonly take to resolve a UX issue? We work in monthly sprints. So we have, if it's a large issue, we will make a specific product discovery issue for a month in advance and work through that issue with product and engineering and design altogether. And then it's scheduled for the next milestone. So then you have a month to implement it. So depending on whether it's a smaller, large issue, maybe two months, but if it's a smaller issue, then you work on it and with the, with your team and then you implement it in that same milestone. How many of us are on the UX team? Oh man, I think there's like 10 of us now. Or 12. Oh, it's on a slide. I know that I, this is the worst question because I have to use my. Okay. All right. Thank you. That's awesome. So the next question, and we only have a couple of minutes. There's so many good questions. Are you open to looking at candidates that have diverse professional backgrounds for UX roles, business analyst, or only specifically looking at UX experience tires. We're always open to transferable skills. Right now it is important though, that you do have some UX experience. We don't really set a time constraint on it. So if you're someone that changed career fields and you've been doing UX for just a little while, but you want to fur your hat in the ring and see, please do. And we will give you feedback on why maybe right now isn't the best time for you to join the team. So it's, there's no harm in applying and, and seeing what the answer is. And that's coming directly from the hiring manager. So you heard it straight from the horse's mouth. So the next one is how do you use customer insights to influence the direction of get lab? That's a really good question. I think that's a really good question for product managers. I don't know Tori. If maybe Tori has some more insight into this. I do know that we, we do a lot of customer calls. We meet regularly with customers. We're often asking them for their input and to look at things that we're proposing and get feedback. And so obviously that feedback does have a really big impact on, on where we go and what we do. But a lot of those kinds of conversations are definitely done in the realm of, of the product management sphere. And it looks like we only have one minute left. So let me see. I mean, if anybody here asked a question that did not get answered, please find me on Twitter, LinkedIn, go ahead and connect, ask me your question, or you can find me at get lab at SSL off is my handle or should be my handle. If I'm remembering it correctly, I'm pretty easy to find though. It is the top of the hour. So I want to say thank you to everyone that attended. We'll go ahead and get this recording sent out shortly so everyone can review. And I hope to see some messages in my inbox. Thank you everyone. Bye bye.