 Hi everyone. Thank you for joining today. My name is Wana and I'm super excited to have been invited by Product School to host this webinar. After I complete my presentation, I'll be with you in the chat to answer your questions. So don't be shy. I look forward to speaking with you. As far as taking notes, I'll share this deck with Product School and they will send it to those of you who registered after we finish. With that being said, let's dive right in. First of all, I think you have two big questions. Number one, why would you listen to me? And number two, why are the right metrics so important? Chancellor, number one, I have eight years of product management experience and I have over 13 years of entrepreneurship, during which time I built and scaled global products in startups and enterprise companies. At the moment, I'm a product manager for Meta and I also love mentoring other PMs and public speaking. So I'm constantly developing myself as a thought leader. I try to post on LinkedIn as often as I can. So watch out for the end of this presentation for my LinkedIn details. And to answer the second question, why are metrics so important? Well, if you don't measure the right metrics, you risk getting three things wrong. Number one is the background, which helps you to build hypotheses. Number two is the results, which helps you to validate your hypothesis. And number three is the value, which is probably the most important of all, making sure that your product is always valuable for your customer and also contributes to the business revenue model. And always, always make sure when you're measuring that you're compliant with data privacy as data privacy regulations become more and more global. And if this is not enough to convince you, before we discuss actual metrics, I want to start with a story. And this is one of Dalai Lama's favorite stories. It's about a Zen disciple who left his village to live in the neighboring forest and seek enlightenment. He devoted himself to being able to walk on water and after many years of practice, he succeeded. One day, the Dalai Lama saw his Zen master walking by. Keen to demonstrate his mastery, his disciple leapt to his feet and proceeded to walk across the nearby river while his master watched. How long did it take you to learn to walk on water? The Zen master asked. 27 years, the disciple proudly replied. You idiot, the Zen master said. For a few pennies, he could have taken the theory. What's the morale? As product managers, it's our job to always evaluate the effort and the rewards of what our teams are working on. And the best way to do this is by using the right data to inform our decisions. And in this presentation, I want to stress the importance of using the right data in the right way. Now, like many of you, there was a time when I was new to product management. And side story, I may have looked like this, although I will not confess whether I had or did not have a pen with a ping-pong-pong. But as I developed through the years, I learned some easy traps to fall into, especially when the business is pressuring you to show growth. So don't make the same mistakes that I did. And don't let yourself intimidated by the business. Let's put these metrics in the bin where they belong. The first metric to go in the bin is the number of logins. Because this will not tell you whether your product provides value to your customer on its own. It can, in fact, show you that they might be getting disconnected too often. Consider using the success of actions taken per user, per time frame or something along these lines. We'll discuss later how to find the right metrics. The second one is time spent. This is highly dependent on the situation, but generally time spent can easily become a trap by wasting user's time. It can even be detrimental to their well-being and enjoyment, and it can build you a bad reputation if you do that. So consider combining the time spent with metrics that indicate value is being provided to your users. And finally, plain number metrics. So, for example, the number of times that a new feature is opened, kind of similar to point one. This doesn't mean on its own that the feature is successful. People are curious, so they might click and you might see a spike in the first phase. But then without context, it's just a number. Instead, maybe log for context either as a percentage or by measuring the number per user, per visit, or in relation to a time frame that you can understand. So I know this all may sound vague. Therefore, in the following slides, I'll teach you how to find the right metrics. So now that we've been those types of metrics that we don't want to use, let's look at the right ways to find good metrics. And I'll go through a couple of well-known frameworks and my opinions about those frameworks, as well as my preferred ways of working. The deck will also have a larger list of examples in the appendix for those of you who have registered to receive the deck from product school. First, I'd like to discuss the most popular frameworks used by product managers and teams to split metrics into types. Can you guess the one I'm thinking of here? Like a pirate, you know? It's kind of a similar sound. Maybe I'm not very good at this, but moving on. Acquisition, activation, retention, referral and revenue. This is a very popular framework to bucket metrics. But personally, I don't like that engagement and the magic moment when the user achieves the desired value are not very clearly represented in this framework alone. So let's look at another framework. Let's make another guess. Yeah, the heart framework. The heart framework was created by Google and it tackles happiness, engagement, adoption, retention and task success, which is similar to the magic moment that I was mentioning. And now that I did my duty and I presented the most established frameworks, let's scratch that and move on to what I would actually use. For a smaller company, I would use a combination of the previously presented frameworks. I would use acquisition, activation, and then in the retention, I would introduce the heart framework and then continue with the referral and revenue. I'm not going to go into too much detail about what each of these buckets mean because you can find information about them all over the internet. They are very popular. And second for a larger company, I would mainly use the heart framework. But how would I use it? There's a caveat because I would like for you to make sure that your data is reliable. What happens is in large companies, they often don't know too much about where your inputs are coming from or how your outputs are being used. And also the data comes from various directions and you might not have a clear understanding of how reliable it is. Different sources might tell you very different things about the same action, let's say. So make sure your data is reliable. This is going to be a key takeaway if you're in a larger business. And now that we've discussed the buckets for metrics, let's see how we can find the best metrics. And this is by far my favorite framework to use for metrics. And it goes like this. First, you put your Pikachu Detective suit on. And then you start looking and deep diving. First, you make sure that you understand your goals. You define the product and the business goals. You write them down. You look at the mission and the overall value for the customer. For example, if we were to take an imaginary example to discuss now, your goal could be to become the best photo service for the elderly. Then you can start thinking what actions the users are taking to achieve those goals and what needs they have during those actions. And this is where the heart classification comes in very handy for some inspiration. Personally, I like to map out the user journey step by step and identify the relevant actions at each step or the relevant needs. In our example, improving the usability for less technically-abled people could be a particular need for the elderly segment and that would be something to be mindful of and to add to your list. Then when you have the list of needs and actions and pain points to use a buzzword, then you can start identifying metrics that will help you understand how you are doing in relation to those needs and pain points that you identified. My tip here is to make sure that your metrics are measurable and don't be afraid to select very specific metrics rather than just generic ones. So be bold, choose complex metrics, metrics that will actually tell you what's going on. So for example, the progress of enabling the less technically-abled people to use our product could be reflected by measuring the percentage of accounts that completed the tutorial at least once. And then you go into evaluation. You evaluate your metrics. What are the pros and cons of each metric? At this stage, you decide on different types of metrics and how you separate them. You decide what your north star is. You decide what your driver metrics are and what your guardrail metrics are. For example, a guardrail metric for our hypothesis here would be the percentage of pages in the tutorial that have been viewed for at least five seconds in order to make sure that people are not just skipping through the tutorial and they're actually getting that upskilled on how to use the technology. And this is a summary of my views, but you can find a lot of information about these frameworks online. And I wanted to focus on my own tips that are generally different from what you would find online. And then since I can't leave you without a summary of the most important takeaways from this presentation, I wrote a list of the five metrics, commandments that I highly recommend you follow, or else things happen. The first one, the first commandment, is to measure what's meaningful, not what's easy or not just what's obvious. Make sure your metrics are clearly related to specific behaviors and they can be moved in a predictable manner. If you make a change, you want to know which metrics you're targeting to impact ahead of time. Commandment number two is train your stakeholders. Make sure they understand why measuring is so important and how they can benefit from it too, so you can have them on your side. Commandment number three is set the right metrics from the beginning, but then it's never too late to change your metrics. Ideally, if you don't have them set right from the beginning, either because you're human and we all make mistakes or because you have a legacy product and they weren't chosen well, don't be afraid to change them. You don't have to work with four metrics. The fourth commandment is to make sure you tell the story behind the numbers. People rarely resonate with metrics and rarely understand what the metrics really mean for your users. And it's your job as a product manager or data scientist, even if you have one, to help others understand the story. A good story is what gets you buying. So talk about the experience, talk about the frustration of your users and talk about the vision for the product and how these metrics will enable you to see where you are on the path towards achieving that vision. And the final commandment, number five, is don't just listen to metrics, talk to your customers. Make sure you try to shadow your customers if you can, because real experience may surprise you and you may find that the metrics that you thought were good may not quite be the metrics that your users care about. And then you end up like in this very popular example on social media that product managers and UX managers tend to like and share very much. So let's not get there. With this, I hope that you will now feel confident to find and use the best metrics to drive the most valuable decisions for your customers and for your business. And if there are any buzzwords or any keywords I used in this deck and I didn't fully explain them, that's because you can easily find them on a Google search and I wanted to focus on my value and my learnings that I wanted to share with you and give you this advice. As I mentioned before, this deck will be shared with you by Product School later, including the appendix with some more examples to pick from. And also feel free to contact me or follow me on LinkedIn. And if you found this webinar useful, share it with your peers. I'll see you shortly in the chat. Thank you for attending. Bye.