 Hello everyone. Good morning, good evening, and welcome to this webinar by Product School on Product KPIs and Metrics that every product manager should know. Before we jump into it, a quick background about myself. Well, my name is Zafir Rais. I'm a Senior Product Manager at Zalando, which is the largest and leading fashion e-commerce platform in Europe. I'm based out of Berlin. And before Zalando, I've been associated with different domains as a product manager in edtech. I worked with Future School of Prime, which is an initiative sponsored by the Government of India for the IT upskilling in India. I've also worked for companies like Cornerstone in the past. Before we jump into the discussion on KPIs and Metrics, let's talk about the synergies between data and product managers. Well, if you're building a product without data, I wouldn't call it a product. It's only an opinion, because most of the things would start with, I think XYZ feature would work. I think this XYZ feature would not work. Well, as product managers, we should not be talking in terms of, I think, we should instead let data do the talking. If data is talking on your behalf, it makes your life easier to convince the stakeholders to get buying from stakeholders. It just leads to fever arguments of product managers between different stakeholders. We all might have heard about this quote, which says, data is the oil of 21st century. But Peter Sonningard has taken it a step further and he calls analytics as the combustion engine. This is beautiful, because just like a combustion engine cannot work without oil, and oil is more or less useless without a combustion engine, product managers should not be working without data or rather cannot work without data. That's how critical data is for product managers. Always strive to be data-driven product manager rather than an opinion driven product manager. I think that's what the difference between create and good product managers lies. Well, data will talk. Data will definitely talk if you're willing to listen to it. If you torch your data long enough, it will tell you the secrets of how to make your product successful. As I said earlier, building a product without data, you're only making an opinion and continuing to build a product without data is the biggest crime in product management. Data and finally, data is a product manager's best friend as I see it. What are the tools to capture? I think this is not really important. Don't get me wrong. These are great tools, Google Analytics, Amplitude, Mixpanel. I've used them myself. These are great tools and it will present you with a ton of data. However, it is useful only if you measure, only if you use them properly, only if you measure the data that is coming out of it. Hence, everything comes later. I think we first need to be very clear as to what we are doing with data. Make use of KPIs and metrics to decide if the hypothesis that you're working on is going to create the right waves, the waves that you intend it to create to have. Now, coming to KPIs and metrics, but what are these KPIs and what are these metrics that we keep talking about? Well, KPI stands for Key Performance Indicator and you have three very important terms that come in here, which is the first one is ski, which signifies how important a KPI is. It shows the importance of this thing that we're trying to measure. The second one is performance. It's measuring performance probably over a given period of time because performance needs to be measured over a time period. Hence, it measures your performance. An indicator which signifies that it's quantitative in nature, which also means that there are a lot of numbers that you need to work with when you're talking about KPIs and metrics. It's a quantifiable measure of performance that demonstrates how effectively you're achieving your main strategic objectives. On the other hand, metrics are measurements of specific activities or processes. They're more tactical in nature. There's something that are measured probably on a daily basis and there's something that are measured on a daily basis and they are more tactical in nature. This is important. Every KPI is a metric. However, every metric might not be a KPI. It's a superset and a subset kind of an analogy, wherein a metric is a superset and a KPI can be a subset of this metric superset. I think a good example of which might help us understand the difference between a KPI and a metric. Let's take an example of Netflix at this point of time. We all know Netflix is struggling with a low subscriber rate. Many of its existing subscribers are dropping off and that's where the problem of customer retention comes in. Customer retention is probably one of the most important KPIs that Netflix is measuring currently and how is it measuring? It's making use of the return rate metric or probably the NPS metric to measure the customer retention. Now, hoping this difference is clear and now we should be good to jump into it and look at the 10 product metrics in KPIs that are extremely important for product success. Now, these metrics could be across products. It is not necessary that all of them apply to your product. It is also possible that none of them apply to your product. It is by no means, this is an exhaustive list of KPIs and metrics. You can have n number of metrics that you are measuring. Why the number 10? I don't know probably because I like the number 10 and according to me, these are 10 important metrics that probably you should be measuring at your end. It also gives you a good head start or an entry point into exploring these KPIs and metrics and how you should be working with them. Having said that, coming to daily active users and monthly active users, well, this should actually be active users and we'll see what that means. Now, the end goal of any product is to generate money. Products are owned by companies and companies are here with the intention of generating profits and earning money and being more profitable. However, to generate money, the primary focus is to get users on the platform. You may not be able to generate the expected revenue from your product or your company if the users are not coming on the platform. Hence, that is important to hear and that is why the active users are metric. However, we do not want any user on the platform. There's a very specific definition of what you call as an active user. Active users are users that perform some valuable activity on the platform. This valuable activity would be closely linked to the DNA of the product. For example, watching a video might be the quality of an active user on YouTube. Sharing pictures is something that an active user would do on Instagram. Similarly, reading a blog is something that an active user on medium.com would do. Active users can be measured in different time frames, like monthly active users or weekly active users, depending on the product and the requirement that it has. DUE, DAU, generally is used for mobile applications because mobile application is something which is a very personal device. You carry it on a daily basis. There are multiple times when you have the mobile in your hand and you're accessing different applications. Hence, DAU is mostly used to measure mobile application products. Online games and social network insights because their usage is more on a daily basis. However, this is not limited and it can be extended to other products as well. The ratio of daily active users upon monthly active users is useful in identifying the stickiness of the product. How do you define stickiness of the product? It's the ratio of DAU divided by the MAU. A stickiness of 20% is normally considered to be a good stickiness score. However, a stickiness of more than 20% is generally considered to be a very successful product. Stickiness also allows to track growth. It's an important measure to track growth or also to track the decline of the product if it's going up or down in either case. Not every product should be measured on the basis of these metrics because not every product needs to be used on a daily basis to be successful. A good example of that could be Uber. I may not use Uber every day but I use Uber on a Friday night when I'm out drinking and I don't want to drive my car. That's when Uber comes in. LBNB, it's a great product but I don't use it on a daily basis. I would be using it maybe once a month or once a quarter when I'm off vacation or when I want to book a stay. Not every product needs to be measured on a daily basis. What is daily active users? Well, it's a number. There's no formula but it's the number of unique users visiting your application during a predefined one day period. The next one is monthly recurring revenue. Monthly recurring revenue simply put is the income that a company can expect to generate every single month. It measures an organization's financial health. It's really important for SaaS business models which are based on a subscription scheme. Amazon Prime, Spotify, Netflix, etc. which work on a subscription model this becomes an important metric to be tracked. However, on a standalone basis, it might not make a lot of sense to a product manager. It needs further analysis. It needs further analysis because this can help you identify where are the customers dropping off and eventually it will lead you to also understand the why of it. Why is this happening? In order to increase the MRR of a product, it's something that should reside with the product managers. The different types of MRRs are new MRR, expansion MRR, churn MRR and net MRR. What's the formula? The formula for MRR is monthly average revenue per user into the total number of users. In case of B2B companies, users can also be replaced with businesses or accounts that you are catering to as a product. The next one is customer lifetime value or CLTV. CLTV also allows you to determine the amount of revenue that can be generated from a user or account in the long run. In the long run would mean the lifetime of the user. Again, let's take an example of Netflix. CLTV displays or defines the average profit generated from one user before they cancel their Netflix subscription. Now, how does Netflix define the lifetime of a user? Take an example of me. If I'm paying 10 euros per month for my Netflix subscription and I keep this subscription on for a period of 12 months, that is one year, my total customer lifetime value would be 120 euros in case of for Netflix. Similarly, it is also calculated. Interesting thing to note is that Netflix has one month as a free trial the first month. That also needs to be taken into concentration where I'm calculating the lifetime of the user. In this case, my lifetime is 13 months and not 12 months. As I said, average lifetime is till the time the user is a paying customer of the platform. Well, it aims to identify how much you should spend to attract a new customer at an early stage based on the profit generated from the user. This could also help companies like Amazon Prime identify that what should be the cost of their Prime membership considering that they are offering the first month as free. So the first month is free. It could also be taken as a customer acquisition cost, which we will look at later. But it's a good measure to identify how much you should be spending to attract a new customer based on the profit or the value that the customer is generating at the end of the period. Again, it helps you identify the right customer acquisition channels. Where should you be spending the money? Is it on social media campaigns or maybe television advertisements or print media, etc. Or should you be giving marketing schemes and discounts and offers in this case? What are the purchasing channels that you should be using? Should you be spending money on Google Ads? And what are the retention strategies that could be used? I remember for that matter when I cancelled my Netflix subscription for the first time, they came up to me with an offer that, hey, we're giving you two months free, please take back with our platform. So that's one of the retention strategies. The formula for CLTV, well, it's the average revenue per user into the average lifetime of users on the platform. The next one is customer acquisition cost. Now, what is customer acquisition cost? Well, it's something which is mainly used by product marketing teams. What it does is it computes the cost of acquiring new customers, including marketing spends, advertisement sales, expenditures, etc. So if you look at its formula, it's everything, all the expenditures combined, the salaries, the overheads, the marketing spends, etc. upon the number of new customers that you've managed to acquire. And now for PMs, for product managers, it is important to know the values as it complements other metrics. How does it complement other metrics? Well, it can also help you identify what should be the free period in case of subscription model. Should you have the first month free? How should you be taking in the money? Is it that I do not provide any credit card details and you enjoy the first month for free or you take my credit card details? I will not be charged the first month, but for the next 12 months, I will be charged all. I will be charged for the entire 13 months out of the first month would be free. So in this way, it helps product managers to understand how it complements other metrics and how can I improve the other metrics? Well, an ideal CLTB to CAC ratio is 3 is to 1. That means the amount of value that a particular customer is generating should be three times the cost that is spent in acquiring the customer. Less means you're spending too much. So if the ratio is less, that means if the denominator is higher. So let's say in this case, the ratio is 3 to 2. This basically means that you're spending too much money in order to acquire new customers and those new customers are not generating the expected value that we thought they would derive. Whereas spending more means you're spending too little and you're missing out on new business opportunities. Let's take CLTB to CAC ratio of 5 is to 1. This basically means that for every dollar spent in order to get the customer, you are generating $5 worth of business. It also means that if you spend more money in acquiring new customers, the total revenue generation could be higher in this case because by just spending $1, you're making a business of $5. So you might want to increase the denominator so that ultimately it increases the numerator in this case as well. Formula, it is the sum of all costs involved in customer acquisition efforts upon the total number of customers acquired as a result. The next one is the session duration. It's one of the easiest metrics to track for digital product usage and what is it? It calculates the session duration of a group of bounced or churned users and this helps me identify how to improve user interaction. It helps you to understand what made the users churn. Now, this could be a certain page. This could be something very annoying on a particular page. So the average amount of time that the user is spending on the platform in one session will give you insights on these metrics and how to fix this. The formula, well, it's the total time spent by users on the product divided by number of users. Bounce rate. Bounce rate is, again, it's a very easy metric to target to track. Google Analytics personally is one of my favorite and best ways to measure bounce rate. It also happens to be the cheapest since Google Analytics is free. Now, it helps product managers identify any page where the users bounce off without spending a lot of time. In my experience, bounce rate really helped me tackle one problem where my signup form was really long and I saw that users are bouncing off from that page and that helped me shorten the form, optimize it and reduce the overall time required to fill the form. That's the way in which bounce rate has helped me improve one of my product APIs. It's also a hint that something is wrong. Well, if you have different entry channels into your product, let's say some of them land on the home page, some of them land on the about us page or some of them land on the sign in or sign up page. If you have different entry channels into the product, it gives you a hint that something is wrong with a particular entry channel because the product is the same. The functionalities are seeing why is this happening that users are bouncing from this particular page. This could be a hint that something is wrong and need a fix. It needs an urgent fix. It actually impacts other metrics like DAU or MRR as if the users are bouncing off from a platform where you will not really have a good DAU or you would not be able to generate the revenue out of that user. It's easy, but it's a very important metric that needs to be tracked. What is a bounce rate? Well, it's the ratio of users, it's a ratio or a percentage of users who visited your product or your page only, who visited only one page of the application before leaving the application. Out of 100 users who come to your platform, if 40 users just drop off after visiting one page, your bounce rate is 40%. The next one is customer retention. This is a rather important one. It is the ability of your product to retain customers after a certain period of time. Let's take an example of Netflix again. It's been a favorite by taking examples, but in case of Netflix, how many users remain or are retained on the platform after the one month of resubscription that ends? Now, that number would give Netflix an insight as to what is their customer retention rate. It's actually a parameter to measure the stickiness of your product. According to Bigg's panel, average CRR for most software products is below 20% over a period of 8 weeks. So that's pretty low. What is it? It's the percentage of users who prefer to stay in the platform. It could be for whatever time period, if Netflix wants to see what's the customer acquisition cost, it would take the time period as one month, wherein you have the free subscription to the platform. Now, what is important over here is you need to define what a returning user is and what your desired time period is. For example, is a mere sign-in enough to consider a retained user? Wouldn't be the case in most applications, but that's something that you really need to identify. What do you define as a returning user? Just the way we define for active users, a returning user is really important. Is the returning user doing any activity or do you call that user retained just because the user is still there on the platform? You might be logging in once in a month, once in a year. That's something that would differ from product to product. A complicated formula, what is CRL? Let's say if I'm calculating the CRL for a particular month, what I would do is number of users at the end of the given period. So number of users, let's say at the end of July, so 31st July, what's the number of users on the platform? That can be 120, let's say the 120 users minus the new customers gained in the time period. So in the month of July, I gained, let's say 20 users. What were the number of users at the start of the time period? Well, let's say in this case, this is 90. So I have 120 users right now, 20 users came in in this one month, and what were the number of users at the start of the time period? That was 90. So this gives you a decent, a good enough customer retention rate, and that's how a customer retention rate can be calculated. It's almost a 90% customer retention rate in the previous example. Okay, moving on to the next one that is churn rate. Churn rate is the exact opposite of the previous one that is customer retention rate. It measures the users that left your platform, not retained but left the platform. It measures the number of users lost in a given time period. What are the types of churns? Now the types of churns could be user churn or a revenue churn, when you're losing out on revenue or you're losing out on users in this case. Revenue churn is obviously, revenue churn is an effect of user churn. If you're losing users on the platform, there would obviously be a churn in the revenue as well. It makes sense to pay attention to revenue churn rather than customer churn because customer churn, although customer churn gives you inside into customer satisfaction. So if you really want to understand what the customer pain points are, how satisfied the customer is with the product, you should be looking at user churn. However, revenue churn is important because that's where you're losing out on money. You're losing out on your revenue. So I think that deserves more focus. The average churn rates are less than 10% of MRR in an ideal case. Churn end at the low end of 2% would be considered good. So if you're losing, if your revenue churn is around at the lower end of 2% of your MRR, that should be a good figure. User churn rate would be the total customers lost upon the total customers. Similarly, in case of revenue, it would be revenue lost upon the total revenue that you generate. The next one is the NPS, Net Promoter Score. What is a Net Promoter Score? So a Net Promoter Score is a survey which is used to measure customer loyalty or user loyalty and satisfaction by asking the users how likely they are to recommend the product on a scale of 1 to 10. So to put it short, there's a scale of 1 to 10 and I will ask the user how did you like my product? If yes, how likely are you to recommend this product to your friends and family? I'm sure a lot of you have filled this for some product or the other. What is the product trying to do with this? They are trying to measure the Net Promoter Score. Now, the respondents are usually classified into three categories. They are promoters, they are passives, and they are detractors. Promoters are loyal product users who will use the product service and actively refer the platform. So if you really like something, you would obviously tell your friends and family that, hey, this is a great product. I highly recommend you to use it. They fuel organic growth. So you do not really have to spend money to grow your product. In this case, it's kind of like free marketing. On the scale, they would rate the product in the survey between 9 and 10. Passives rate the product 6 to 8 and like the product. However, they are prone to shift their loyalty to a competitor if the competitor is able to pull them towards itself. Whereas detractors rate the product between 0 and 6 and in general are not satisfied with the product. A good example in this case could be of Apple. Apple probably has a very high Net Promoter Score because there are certain users who are hard for Apple loyalists and who would be the first in line whenever there's a new iPhone launch or any new Apple product which is launched. So Apple, for example, has a very good Net Promoter Score. Detractors can do negative publicity as well. And this will eventually hamper the image of the product. Again, a good example of this could be the case of Samsung where phones, I think it was the note where phones started going up in flames that caused a lot of negative publicity and your passives quickly turned into detractors in this case. NPS generally lies between minus 100 and 100 and negative NPS scores means detractors are more than promoters. Obviously minus 100 would mean all the people that you surveyed are detractors and 100 would mean all the people that you surveyed are promoters of the product. Some stats in 2018 Netflix had an NPS of 64, PayPal had scored 63 and 54, 53 and 49 for Amazon, Google and Apple respectively. Be polite in timing the survey. So it's really important to ask them to ask the survey at the right time. So for example, you do not want to ask the user to fill a survey when they're in when something has gone wrong. So ask the user if you would want to refer when they're successfully placed in order. Do not ask them when the order placement has failed. That would lead to a bad score. Also, you want to ensure that you're getting genuine responses. So try to keep the survey shorter because the longer the survey, the more annoying it gets for the users to fill a long form. Keep it short, keep it short and sweet and place it strategically. The net promoter score is the percentage of promoters minus the percentage of detractors. Now the last one and probably one of the most important metrics that companies and products would track not star metric. The not star metric is what's first of all was not star. The not star is the anchor of the northern sky. It's a landmark or a sky marker that helps those who follow mostly sailors who follow a determined direction as it glows brightly to guide and lead. So it helps those who follow determine the direction as the star glows brightly to guide and lead. It is the most important metric of the product. True. And what's how did it start? So taking a cue from the definition of not star or entrepreneurs on L is coined the term not star metric for the first time in his book hacking growth. Now this was done to reduce the administration around it and to have a singular goal for the entire company or for a department. It defines the success of the product or the company. So what does success mean for a product or a company? It's linked to the vision of the company. So for example, Facebook. Facebook has a vision of connecting people and making it a more social place. So it would be interesting to know what the not star metric of Facebook is. Again, we'll take an example of Netflix. I promise this is the last time we will be hearing Netflix in this webinar. But Netflix's vision is becoming the best global entertainment distribution service. Thus, we can safely say that the not star metric for Netflix would be session duration as it denotes that users are spending more time on Netflix because the entertainment quotient of Netflix is really high. There is no defined formula for this. It's more of a strategic metric which the team needs to come together and have this metric so that the rest of the team can follow it. Finally, we have gone through 10 important metrics in KPIs but just wanted to leave you with some thoughts. This is just an overview. These 10 metrics, these are not an exhaustive list as I mentioned before. The purpose of this was to introduce you to these metrics. Hence, it's more to help you inculcate a much more data-driven attitude. It's not exhausted. There are many other metrics that might be important for your product. But I think this is a good starting point again to become a more data-driven product manager. Dive into the details of each one of them. Look for metrics that might be relevant to your product and to your organization. Well, we cannot do a deep dive in this webinar because each metric would then take a separate webinar in itself. What are the next steps for you? The next steps for you could be identify the metrics for your product and start measuring it. Also, you can also think of some popular products and think of what their not-star metric could, sorry, what their different metrics that they measure. A good example would be a not-star metric. This is something that can also be used in cases of interviews for product management wherein you can ask the interviewer, hey, what's your not-star metric? Or according to me, this is the not-star metric. Is my thinking in alignment? I think that can lead to a good conversation in your interview as well. Finally, don't just track metrics as important it is to measure and track. It is also important to make use of the data that is coming out of this. And remember, build a product not an opinion. Finally, I want you to leave with this quote from Mark Dwayne and he says, data is like garbage. You better know what you're going to do with it before you collect it. That's why the product will come in later. You need to know your metrics. You need to know what you want to measure and why you want to measure. And that will make your life easier. Thank you so much for spending time. If you want to stay connected, you can reach out to me on LinkedIn. I'd be happy to answer more questions on this topic or any other topic in general. Thank you so much. See you later.