 All right. Hello. It's working. Good to see everyone. You're here late on a Friday at Slush. Most people are getting on a plane. Excited to be here. And today, we will talk a bit about some of the best practices for engineering product and design teams. And I talked to a number of our portfolio companies about this and tried to collect the biggest challenges that they experienced both at the early stages, pre-seed and seed, and then also through growth. So the goal today is going to be to talk about strategies that are going to become more important as companies grow. Obviously, at the earlier stages, things are lighter weight. And when you're the size of Google, these processes will still be productive. So a few key points. We'll talk about how the teams look. We'll talk a bit about OKRs, which I assume most people are familiar with, but try to layer in some best practices that you might not have heard of. And then some product planning practices that tend to help teams, especially as they're growing, balance what each constituent of the team is working on. So what do EPD teams look like? The obvious place to start. There's a product manager, a design lead, and an engineering manager. I think most people here are probably familiar with the two-pizza rule of kind of keep the teams relatively small, so less than eight people. Tends to be a good initial best practice. I think one of the things that I hear very often from companies when they're structuring this, and I'm seeing the cameras come out, all these slides will be available on our website. There'll be a link at the end, so don't worry. There's a lot of detail on them. We won't go into everything today, but you'll be able to go back and look at it later if you're interested. So looking at the role of a product manager, I think, especially within Europe, there's kind of this dichotomy that has emerged around what is the difference between a product manager and a product owner. So I thought I'd spend a little bit of time on that today. The baseline difference is that a product manager is a role on a team that inherits responsibilities at an early stage startup that typically start with a CEO, which is, what is the big need that someone has? What is the product that I am going to deliver to that person that's going to help them address that need? And then how am I going to prioritize the aspects of that product? The features in an order that is going to best test my hypotheses for how I'm going to address this need that I've identified. And as a company grows, and a CEO's responsibilities change from kind of focusing on building the product in a 3, 5, 10-person team to fundraising, hiring people, setting culture, managing a team, managing key stakeholders, sales, stuff like that, you need to bring in product managers that are able to hear from the leadership, OK, what are our goals? What are our values as a company? What do we want to achieve? And then funnel those into a product strategy and a roadmap and a set of priorities that the rest of the team can then ingest and feel like they're well-informed and able to make efficient progress against. So product managers are really taking that piece of a founder and CEO's brain and institutionalizing it within a company so that the CEO doesn't have to do that job. And obviously, as a company scales, you have many of those pockets focusing on their areas, and they all kind of roll up. So that'll be a theme that we keep coming back to here. A product owner is actually a hat that you wear in an agile scrum process. So if you kind of look up where that word comes from, it's actually not a role in a company. Like, you don't hire product managers in the same way as you don't hire a meeting minute taker in a meeting. That's a hat someone in the meeting wears for that particular meeting because someone has to do that set of responsibilities. And typically, a similar person will do that in every meeting. And similarly, in a lot of scrum processes, the product manager will wear that hat and do that set of responsibilities, but it's not holistic. You're not able to do everything that a CEO would need a product manager to do if you're just doing product owner responsibilities. So the first big takeaway here is to set up EPD teams up for success and to avoid a challenge that some of you might have already experienced. Sometimes it's around late seed or early series A where the product manager or product owner isn't thinking strategically and isn't taking on that burden of figuring out what should we do, not just how to execute what I was told to do. That's where finding someone who thinks like a holistic product manager is invaluable relative to someone who might have taken a product owner course, again, for that subset of responsibilities. OK, so then with that in mind and kind of thinking about what a holistic product manager is next to engineering and design, OKRs is something that I assume, raise your hand if you've heard of OKRs. OK, like most people will go into what they are, but it's kind of an interesting story. So OKRs are actually pretty old. We think of them as this new framework, but it was around during the time of Intel, Andy Grove kind of at Intel published a book and that's when this was first mentioned. And then John Doar, the famous venture capitalist from Kleiner Perkins, was at Intel at the time, was like, this works pretty well for a big company. I think it would work for a small company too. So he suggested it to a small startup in his portfolio called Google. And Google used it as a very small tech company and kept it all the way through now when it's hundreds of thousands of people. And that process was demonstrated to scale really effectively. So a lot of the people that left Google and started other Silicon Valley startups basically tried variants of this and all kind of converged on the fact that OKRs really effectively help leadership communicate down through a company what to focus on, which ends up being a big challenge especially during the hyper growth stages of a company when you have a lot of new people who are just getting integrated into the culture and are trying to figure out what to work on. It also works for a very small team. You could be 5, 10, 15 people and you make a little whiteboard or notepad of what do we wanna focus on for this quarter or this half and kind of fill it out in a very lightweight way. So you can do this in a lightweight way or a much more rigorous way as Google does where they have a whole tool for it. Okay, so how do you do this? What are kind of the myths and what's actually useful to do when implementing OKRs? We'll try and touch on it for very early stage companies and then also growth stage companies. So the objective is the what? What do you want to achieve? Do you want to increase retention? Do you want to lower acquisition costs? The key result is how are you gonna measure success? Like what is the precise metric and then what is the target? We want to increase retention from 20% to 40%. Okay, that is a precise, measurable key result and then you can evaluate your success against that. If you look at Google as kind of the golden standard, which you may or may not, I think they do a pretty good job. They have a one to 10 scale. They call it zero to one because they're nerdy engineers but it's basically a one to 10 scale and they target success at around 0.6 or 0.7 and the reason they do that is to always have a bar that's higher than what the leadership thinks the team can achieve and then they push everyone to try and achieve that and you're always motivating people intrinsically to kind of shoot farther than you think you can actually go, which is a really powerful element of culture building because you're setting the expectation that it's okay if we fail, it's okay if we don't get all the way to the end goal but we're gonna really push ourselves hard and in order for that to work, I'll go to the next slide for this and I'm gonna skip to number three here. You can't use it as employee evaluations because you're telling people it's okay to fail, it's okay not to achieve things 100%, right? So OKRs are really again a tool to communicate what are our aligned objectives? What do we wanna focus on as a company? How do we make sure we're measuring that in a productive way? How can we look back and determine if we did a good job or not? It's okay not to get there 100% but be aggressive and try for it and then employee reviews are a separate process that you treat separately and that creates a positive incentive structure. This is true of small companies as well as large companies. In terms of the first two points on here, making sure you are actually using it effectively to focus comes down to not doing too many things at once. So three to five objectives, three to five measurable key results, that's it. Even at Google, it's five and five, like it doesn't get bigger than that, same at Facebook or anywhere else you go. For an early stage seed startup, it might be two, it might be three. It's okay, like it's better to focus the entire team on a few things and do them really well over a quarter or a half than spread every one too thin and do everything poorly. Like that's when productivity tends to slow down. The last thing I'll mention on here and this is one that I get a lot of questions and debate about is that OKRs should flow down. I think if you accept the underlying assumption that this is a tool to align people and to focus people, there does need to be a starting point where someone in the leadership team, the CEO says, okay, these are the things that are important to the company. Here's what I want everyone to work together to achieve because we don't want to run in different directions, we want to pull together. And if that's the case, you can have ideas that flow up from the bottom and that's a good thing. You want to solicit a lot of feedback and input from the teams. It shouldn't be a dictator, but ultimately someone has to drive the stake into the ground and say, okay, these are the three objectives I have for us for the next six months, like that flows down through the teams and then each, the head of marketing, the head of engineering, the head of design, et cetera, say okay, in order for me to support your goals, here are the goals I'm setting for my team. Here's the part we're gonna play in this team sport. So I think these four things, if nothing else, tend to lead to OKRs being adopted and retained very well and an effective alignment and focusing tool. So I'll leave those there. There's obviously more, but I actually think simple is better than complex for this. Talk a little bit about roadmap planning. That's the second big question I tend to get from a lot of startups at various stages. I'll caveat this by saying, if you're a pre-seed company or a seed stage company, you're probably focusing on one thing and that's product market fit for the singular idea that the founding team or the CEO has, and that's okay. You don't necessarily need to do strategic planning at that point because you're just testing a singular hypothesis. I think when that product market fit is achieved and you're now iterating to optimize your product and improve certain metrics, or you're introducing new products, maybe you're a series B stage company and you're starting to test other accretive bets, then this process becomes far more valuable. So I would always start very lightweight. There's gonna be a lot of stuff on here. I wouldn't recommend implementing all of it, but it's good to be aware of it because as you see success in the back of your mind, you can kind of know what are the tactics that you can take in order to set a culture that is gonna be ready to adopt kind of more and more rigor on how you're getting larger teams to work together. Okay, a lot of words on this slide. You don't have to read it or take a picture. It'll all be available later, but the key things to highlight here are kind of the four steps that a good EPD team should go through, regardless of the size of the company, to make sure that they don't make, that they have the lowest likelihood of making mistakes. The first step, and this is typically either the CEO in a very early stage company, or a product manager in a company where the CEO is now building a function to do this part of the strategic work, is to prepare. And we'll talk a little bit about a five C analysis as a method to do that. This is to say someone needs to gather the insights, whether that's user research or market research, et cetera, and bring that into the company and say, okay, here's the information that we're gonna need in order to make smart strategic decisions. The second step is to prioritize impact. So you have likely a laundry list of good ideas, ideas from the engineering team, ideas from your competitors, ideas from your CEO, et cetera. And based on the OKRs, based on your company's objectives for that quarter or that half, the order of that list is gonna change, right? If one quarter you're focused on retention and then you achieve it, and the next quarter you're focused on lowering acquisition costs, the features that you prioritize are going to change. That's part of being agile and building your product. So having someone go through that long list of good ideas, all of which are probably good in some way or another, and prioritizing them is really valuable. And the PM should own that. That's an example of a set of responsibilities that a product owner doesn't have, right? Because it's not within the scrum process. This is strategic thinking that is in addition to product ownership responsibilities. The third step is to scope the effort. This is where the whole core team comes together, and this is where having kind of the two pizza rules important. You don't have too many people involved in this process, or else it just gets too messy. And you scope the effort. You say for the highest priority activities or features within this finite period of time, a quarter or half of a year, okay, how much work are these different ones gonna take? Because now we're going to factor that into a rebalancing of those high priority items. And I'll show you what that looks like in a second. And then finally, and some companies forget to do this, is you have to synthesize all of that work into a set of internal communications that you share with the team. People have to understand why you're doing what you're doing. They have to understand the impact that you think it's gonna have. It's very motivating to know the value of putting in the effort to build these things. They have to, it typically helps, for especially engineers and other teams, to understand that, or to feel like the effort required on their part is appreciated. So even communicating an understanding of the effort required is valuable in terms of motivating people and getting buy-in across an organization. And then also to just capture in a visually stimulating way where people can look at your roadmap or they can look at your strategic direction and say, this is going to be, this is gonna lead us to a place that I'm excited to go, right? Teach, show people, teach people to yearn for the sea and not how to build a ship, that kind of attitude. Okay, so for the first part, this is the synthesis part, or sorry, this is the discovery part where you're pulling in information from different perspectives. I think the 5C method is a great tool to make sure you're not missing a lens through which to look at what options you have available and what constraints you have on you for which features to build and what bets to make. So the first lens is the company's perspective. What is valuable to the company? What are your OKRs? What is the culture of your company? What is your brand? Like what are the different considerations when you look internally that should go into deciding what features you wanna build? The second is who are your collaborators? And this one's often overlooked. So if you're Netflix, right, who are the distributors of your content that you need to be conscious of when you're deciding what products to build, right? So that might be different mobile platforms or different operating systems on laptops or TVs or whatever. So really thinking about, or your content providers in a lot of cases for Netflix. So I think thinking about the ecosystem that you have to be friendly with. Customers and competitors, I assume most people are very familiar with. So I won't spend much time there, but really thinking about your user needs. What do they care about? What's gonna be valuable for them? What problems do they have to solve? And then what are your competitors doing? It's great to look at what ideas have they had. What are they doing well? You know, a typical SWAT analysis, in fact, or that in. And then finally, what's the climate? And this one used to not get a lot of attention, but starting with COVID and now with kind of the economic situation that we're in, it's getting more attention. So it can include trends. It can include, again, macroeconomic environment. It can include policy and regulation. If you're Uber, you're spending a lot of time looking at what are the different regulatory considerations when I choose what markets to move into or what products to build. So these five lenses tend to be a great, just basic framework of pulling in this information, writing it all down. It's great to provide it to the team so they understand all these considerations. And it starts to illustrate the value of a product manager on a team. Like they are the person pulling all this together so the rest of the team can benefit from it. Like an engineer or sales person aren't going outside, like asking all these questions necessarily. They might do one or two things, but the product manager really is responsible for aggregating this. And again, if you're at a very early stage company, the CEO did this, likely, hopefully, when they started the company. So that's kind of the initial product manager of the company. As the CEO doesn't have time to do this for every feature, as the company grows, that's one of the main reasons a product manager starts to step in. The next step, and I've kind of combined steps two and three here, is to assess impact against effort and really understand where do you want to spend your time as a team. So I'll skip forward to this slide. I think this two by two matrix is pretty self-explanatory. So I'll go through it quickly, but I won't dwell on it. I think if something has a huge impact across all five of the Cs, or some of them, and is low effort on your part, that's an easy win. You should do those all day long. That's great. It's not going to take you much work or time or effort, and it's going to really make a difference to people. So you want to spend a lot of time there. I think if the impact is low, but it's also easy, that's kind of an incremental improvement. You're really tweaking this feature. You're nudging retention slightly higher. Facebook does tons of these every day where they're A-B testing different changes to their algorithm and they're seeing how it affects engagement and retention and things like that. So these little incremental fixes can be good. Again, if it's aligned with the objectives of the company, you don't want to do it if it's not supporting the OKRs that you have for that quarter, because then you're not moving the right metrics. Big bets are where it gets interesting. So if it's a lot of impact, but it's also a lot of effort, you have to figure out, do you want to do this or do you not want to do this? This is where a 5C analysis plays a huge role, because you really want to study, OK, what are the effects going to be across all of these different constituencies? And because it's going to take so many resources, whether it's time or capital, you really want to have done a lot of analysis before you just dive into it. So you might not do a full 5C analysis for a little incremental change or an A-B test or a feature bump or an easy win. But if you're going to tackle a big bet, you probably should research it a bit before you have a giant team swarm onto it and spend three months building a big feature. And then finally, obviously, avoid resource strains. It's not moving the need a lot. And it takes you a lot of time and energy. Don't do it. Lower the priority of that. So going back to the original slide, again, you prioritize against your OKRs. You scope the effort with your team. You go back and adjust the priorities based on this metrics. You say, OK, now I have a few items at the top of the list that we want to focus on this quarter. And here's how we're going to measure success. Ladder back up to the company's OKRs in general. OK, so some practical frameworks you can use for this. I'll give you a simple one and then a more sophisticated one. I think impact-first effort or value-first effort, I've seen them talked about both ways, so I included them both here. You can do something as simple as five stars for a lot of impact, three Xs for a lot of effort, whatever you want. Doesn't have to be complicated. But starting to list all of those prioritized features you have and then quantify how do I feel this sits on both of those axes. And then, obviously, you sort. And there's not one right answer. But you can scan through it with your EPD leadership and say, OK, which of these items do we want to circle? And say, we're going to focus on those features this quarter. And which of these items do we say, you know what? Now that I've thought about it more, too much effort, not enough impact, we shouldn't do that. We might like it. It might be something certain people are passionate about, but it doesn't make sense. Because at the end of the day, focus is what separates companies that are highly productive from companies that spend a lot of time and don't make much progress. The more sophisticated framework, and many of you might be familiar with us, is called Rice. And it's a more quantitative way to summarize this. So for people who are quantitatively inclined, this is a great tool to objectively look at what should the relative priority be of the different good ideas that we have. And again, most ideas are good in some way or another. The first part is reach. How many people, let's just say it's a feature for Uber, right? So you want to introduce car share, like ride sharing in Uber back in the day, as opposed to just the black cab. OK, what's the reach? How many people are going to want to car share? OK, hopefully the PM did a bunch of work and has some sense. OK, let's note that down, high, medium, low. Impact, how valuable will car sharing be to the people that it reaches? Are people going to say, this changed my life? Or are people going to say, this is nice to have? Confidence, did I do a lot of research? Do I really feel confident that I understand the reach and the impact? Or am I guessing and not quite sure? And finally, divided by effort, which is the same as we talked about earlier. How much work and time and money is this going to take? And I've given some examples of how you can actually put numbers against this at the bottom, but you essentially have a list. You calculate that. You can stack rank it, and then you can say, these are the items that were rated the highest objectively. And then your EPD team should take that as an input. I wouldn't just rely on that. You should use subjective features. You should use intuition and gut feel and soft factors that aren't going to be included in something that is like as simple as this actually is. But it's a really good way to look at the problem objectively, in addition to kind of our natural subjective lenses. And then finally, once you've done that, again, you have your list. So say you have 50 features. You have 50 ideas that you pulled together from suggestions from the engineering team, suggestions from the CEO, things you saw your competitor do, random ideas that the person who helps clean the bathroom came up with, wherever the idea comes from, it doesn't matter. And it's actually great to be inclusive. And you've gone through this rice framework. You now have a prioritized list. Now you need to indicate to people, OK, what priority am I actually assigning to this? Now that I've done the analysis, P0, this is a framework that is used pretty universally in Silicon Valley. So I recommend it because people will likely have heard of it. And it just makes onboarding new folks much easier because there's a standard. It's also very simple, priority 1, priority 2. It's pretty intuitive. So a priority 0, P0, would be a must have. Our product will not work unless we fix this thing. Maybe Apple changed some thing in the app store or whatever. Like there's some new regulation. We have to do this, or a product won't work. Bubbles to the top. Don't have a choice. P1 are the things that are the highest priority when you look at impact versus effort. These are the things we really want to focus on this quarter. And that's what ultimately gets loaded into JIRA, or whatever project management tool you use, and gets broken down into tasks that the team then takes action on. P2s, this is nice to have. If we have time, or the availability, or it's very accessible to us, we want to keep our eye on these balls, but we don't want to prioritize them. We shouldn't work on P2s if there's a P1 that we can be helpful with. But if there's some engineer that really isn't going to be useful on a P1, great. Now we know what some of these kind of backup tasks are. And finally, P3s, which are really backlog. These are great ideas that we're not focusing on right now for whatever reason. We don't want to forget about them, but we also don't want to spend our time working on them right now. OK, so in quick summary, I think we covered a lot. You've been patient, because I know this stuff can be a little bit dry. But when you're forming the EPD teams, this one is non-here. But I highly recommend, again, hire product managers. Don't hire product owners. Focus on people who are going to think strategically, in addition to wearing the product owner hat. I've done a talk on product school about this if you want to learn more about the difference between the two. But I think that's step one. Use OKRs to align your organization. If you're a seed stage startup, just Google, like Microsoft Word doc or Google doc, bullet points. Here are the three things we want to focus on. Here are the three KPIs we're going to measure it against. Done. It can take you an hour with your team. And then everyone really knows what good looks like and what you're going to measure to determine if you achieved what you wanted to achieve. Obviously, as you grow and scale, there are entire companies that make products to manage OKRs and larger organizations. But I think that's probably for a different audience. Number two on here, I would implement a company-wide product planning process. Once you start having product managers, I think if the CEO is still the product manager, they probably have a pretty good idea of what they want to do. And they're doing all this stuff themselves. You don't need a process so much because they can kind of do it internally. I think when you start hiring a team that you're moving that responsibility onto, it's really good to have a process that is embedded into the culture of the company that is going to be able to scale effectively and sets the right expectation and tone, especially for people you're hiring in. So it's smooth sailing all the way through growth. Number three, use the five Cs to inform your strategy. That's really making sure you're looking through all five lenses that are valuable when figuring out what you should be doing, so what the company's needs are, what the customer's needs are, your competitors, your collaborators, and the context or climate. And then finally, impact versus effort. You can use a quantitative method. There's qualitative stuff you can do discussing what the teams are really stack ranking all the good ideas that you have and figuring out what are the ones that are going to be most valuable this quarter and then assigning them a priority level so that everyone in the company understands what that priority level is. And if they go back and look, they can also see, OK, what is the five C analysis we did to inform that? Why do we feel this way? What are the OKRs? So I know the metrics that we're trying to move by prioritizing it in this way. That'll build a lot of comfort and a lot of understanding throughout the team, all the way from small teams to very large teams. So with that, I invite you all. This is all available on our website at lakestar.com slash best practices. Deck looks a little different because I made it darker for Slush because it's the theme. But it should be all the same content. And you're all welcome to reach out to me if you have any questions. That's my time. Thank you.