 I think it's going. So here's a quick introduction about myself. So my name is Nathan. I'm primarily a product guy working in startups for about 10 years, overall, 14, 15 years into software development. I'm more into product management. So in my current role, I lead a team of product owners at a real estate startup called commonflow.com. Also a runner into lean and agile for a bunch of years. This is what I'm going to cover in a quick way. A very, very quick introduction about idea to scale. I mean, how do you just take the whole idea from idea stage to a scale startup? What is the overall process? What defines a good metric? So once you get into the data, how do you take a call on metrics? What are the different types of metrics that people are tracking? Give you some samples, some anecdotes. We'll talk about a bunch of lean and agile frameworks that people follow. You could pick any one of them, and everyone pretty much talks about the same thing. And then how do you sort of structure that to start tracking your own startup growth and various, one key thing out of the whole session is the whole concept of one metric that matters, which is a really, really hard thing. Essentially a lot of people tried, including me, we have tried at a bunch of places, but it's really hard to do. We'll just talk about that. And how do we look at the overall tracking approach? And then a quick Q&A. So obviously, all starts with an idea. Now, it could be a business idea. It could just be a feature request. It could be a new service. You could pretty much apply analytics into any of these aspects. Obviously, you have tons of ideas. Time doesn't, you talk about brainstorming, you'll get hundreds of ideas. The real hard thing is how do you prioritize them? How do you take it from an idea to an actual validated product to a scalable business? So the very primary thing into Lean Startup is the whole Steve Banks customer development model, which I'm not going to go totally too much into it, but it starts with a product discovery. So you start, once you have an idea, you try and validate that idea in your market. So you go through a whole product discovery, try and validate with your actual customer, get out of the building, validate your idea with a set of customers. Once you have reached a product market fit, which is really, really crucial milestone in a startup journey, that's where you start looking for efficiency in scale. So if you really have to draw a life cycle for a startup, it is clearly before product market fit and after product market fit. And just to achieve it, just to tell you the stats, 80% of the startups are still into product market fit, haven't reached that level. So scale and all that is like a totally different ballgame. So, and that's where metric plays in. Like a quick, so there's a really good website called funders and pounders. If you really into startups, you'll really find very useful stuff. It sort of draws the whole picture of how do you go from, typically you hit up on an idea what is really missing somewhere, what is the problem that you hit up on. You try and make a quick prototype, you call it MVP, you call it prototype, there's a bunch of ways to take a look at it, show that prototype to 100 people, iterate, keep iterating on the whole prototype. A lot of people also start finding, if you're a functional or business guy, you start pairing up with a technical co-founder, so that you really start working together and collaborating. Then you start looking for your initial seed funding and stuff, launch it, get some thousand users. One of the key metrics that people follow is that if you, how to know that you're really doing well, each startup, you pick up a key metric and you really have to be growing at 5% per week on that metric and you really have hit a scale and got a million user base and stuff like that. And then you keep iterating on that. So it just sort of summarizes how you go. You're not going too deep into it. It's more of the lean startup 101 than the analytics. We'll go a little into the analytics part. All right, so typically this is, this is what you'll see very often. This is the loop that you go through if you're working for a startup. You have an idea. You start building onto it and during building, it could be prototype, coding, whole bunch of things. As you build, you continuously iterate through and measure what you're building. Is it working, not working? You learn from it, you collect a lot of data and then you learn from it and then you put it back to the overall loop. The successful startup iterate through this loop dozens of time during a week on every feature request and that's where you sort of start getting attraction. The issue with this is, obviously, when you talk about ideas, there are tons of ideas to do. Even within, the moment you start on an idea, you start getting dozens of ideas of how to scale that idea. So it's really become really hard to prioritize what you're doing. Obviously building comes really easy, so therefore we see a lot of products being built which go to customers only after it is built. Obviously, and then it bounces back and you don't know why it is not people are not signing up. We hear a bunch of stories around that. The real issue becomes in the last, the second stage is measured. So if you're not tracking the real metrics which is meant to be tracked for you, obviously the whole loop, the whole learning will be very, very skewed. So you will probably think that I'm just growing my startup, things are looking good but obviously you're tracking the wrong metric. Therefore, obviously you can't make out that you're making a lot of errors right there. So we're gonna focus right there on this stage and skip through the subject. All right, so this is a quick definition of what the analytics is. Typically once you've figured out a business goal which is, this is what I'm gonna do in the startup. Analytics is nothing but it'll try and measure you how closer you are to that goal and help you in that part. For a lot of funded startups typically, this is the whole definition is that you're gonna iterate through the whole product market. You have to achieve a product market fit before you run off money. If you have run off money and you've still not got product fit, I think if you attended some of the sessions, tiny hour and whole bunch of things, you have still not got a good product market fit and obviously the money is gone. So you have to really iterate through the whole loop faster so that you get product market fit and then you can start scaling and looking at the different challenges. In a bigger company, the analytics typically helps you to present the data that you need to, data or buy-in to get a buy-in. I'm gonna do this because this is the data. Otherwise, all you have is just opinions right there. So quick basics on what makes a good metrics. Typically, you have to have a metric which is comparative. So if you're looking at X number of users acquired, it's no good, you have to look at it whether it acquired, is it better than the last week or this week? So you need to start comparing the metrics all the time. If you've got growing at what percentage and from what baseline. So typically, a comparative metric is really, really good rather than absolute number that you track. It has to be really simple because a lot of start-up get into a very complicated definitions and then you spend a lot of time and most of the time to explain what the metric is rather than actually tracking and discussing that thing. So you have to be really, really simple. Keep a kiss principle right there and we'll go through some of the metrics that we used to track. Sure. So the only issue with it is that as you start iterating through it and you're computing a bunch of things and you have a team, you'll spend more time into telling them what that metric is, what we're learning from it rather than actually focusing on the metric itself. And I'll tell you some examples from what we have done and how it sort of changes the whole team behavior and picking up what Calpe said in his talk. Some of them you'll find right there. The other thing is ratios. So you hear about a lot of, we have 100,000 users but that doesn't do any good. If you don't start coming, whether you're doing good or not because cumulatively you're obviously making it from 1,000 to 2,000, 3,000 but is it 10% more than the last week or not? You don't know. So unless you get into ratios, it's not gonna do any help. One of the key principles that typically people say that a good metric is what, if it is going down or going up, it really has to change the behavior. People have to jump out of their chair and say, you know what, something is wrong. Otherwise, you will go through the whole weekly metric reviews and stuff every week and yeah, yeah, nothing is changing. So you really have to pick out one of those which sort of changes the whole behavior for your company. So it's a quick dashboard I picked up from one of these companies. It's a little bad but it basically talks about what is the average revenue per user last month versus this, what is the average subscription? So a lot of them are sort of ratios. Some of them are numbers but they are also comparing positive negative trend from previous weeks. And we'll go through some of the examples right there. So that was about sort of what makes a good metric. Now what are the different types? So obviously you have qualitative, quantitative metrics which talks about numbers but you also have a lot of qualitative metrics which talks about your actual customer discovery which is happening. So if you're doing a lot of customer interviews, you're getting input from them that usability is not good or I'm finding this to be a little difficult. So you really need to make a balance between sort of quantitative what you're tracking and qualitative metric and sort of start talking to customers. So you don't need to just rely on some numbers. A lot of these numbers will tell you what is happening but they will not tell you why is it happening. To get the why, for example, you get Google analytics and a lot of metrics are there but unless you actually talk to the customer that cannot replace sort of digging up. So one of the things that we have done in our company is that we used to analyze a lot of metrics. For example, if you're going through app installs, so you need to track why app installs are going well and really what did we do in the last week that app installs have shot up. Now you can try and figure out with the team why it went up but at the same time you also have to figure out what customers are really liking about it. And suddenly if there's one feature that you shipped that people have started liking, therefore you can sort of write on to it and do more of it. Similarly, if it is going down, you need to figure out what is not working so that you can sort of talk about it. Absolutely, absolutely, absolutely. Changing the behavior, I mean, as we said. Absolutely, absolutely, absolutely. So that is about quantitative quality and the similar is vanity and actionable. So the other good definition is obviously a lot of companies, so vanity metrics typically is more for PR. You know what, we have reached 100,000 downloads. I mean, it's good to get into the PR but you know the actionable are something that is internally you know, dude, I need to improve on this, otherwise I'm screwed. So that is actionable metric for you because otherwise 100,000 downloads is doing nothing. Nothing really there. All right, quick examples of people still track page views. You know, obviously it's a good, so one of the principles is it's not that you shouldn't measure anything, you can measure thousands of metrics, but what should really matter is a very few which gets you into some action. Otherwise it's purely a vanity metric. You have unique page views, you put more money, you'll get more page views. But is it really gonna change your behavior? May not, right? So therefore it's all vanity. Again, one of the other things that typically happens is obviously what you're doing, you're reporting a bunch of metrics, but at times you also end up into an exploratory metric and figure out, all right, we are looking at this pattern and let's dig into the data and figure out if there is something really happening there. That actually helps you into correlated and causal metrics. So correlated is typically, and I'll give you a good example of ARBNB, right? So correlated, typically if you have two metrics that are working in tandem, for example, a good photograph led to more bookings at ARBNB, right? So initially when they started, they were just looking at there's some correlation. Every time we have more pictures coming in from a specific city, there are bookings that are going up. So you get into a correlation and once you are, you start finding, is there anything that is causing there? So the best discovery, think about a startup, if you're able to figure out a causal thing between two, that's where you're able to push that thing and do more of that. If not, I mean, you know, I did something and some of it might have changed, made the impact. So we made a release and it suddenly jumped up or installed. So you know there is a correlation between what you did and what happened, but you're really not sure. You could be having a correlation with a bunch of metrics, but if the more you dig into a correlated metrics and you see what this might have caused this, this specific thing, then you can do little more of that. So when you once you're going through the data, you really have to figure out between correlated and causal metrics and keep sort of doing more of them. So one of the examples, I think, sorry, I already gave the Airbnb example. So we also did at Comfort, we did one of the things, I mean, I don't know, we launched something called a 360 degree virtual tool. So, you know, we work in a real estate domain. So we came up with a cool idea that, you know what, we're gonna change the way people are hunting property and we'll give them a 360 degree views of the whole property. We did a bunch of pilot, you know, with some five or 10 properties and then we started to analyze the data and we obviously put a hypothesis right there, you know what, I mean, we're gonna do this, but is it gonna change? What is it gonna change? So we sort of put a hypothesis much before and I'll talk about that whole framework. And then we realized that the average time people are spending on that is much more. Now, that is just a sort of correlation, but we don't know whether that is the causing it or there's actually generally more people are coming and buying that. And then we observed it over a period of time and found that there is some sort of correlation, but exactly not the cause. So cause typically came when we actually spoke to the guys and figured out what they really like this and therefore I think if you do more of it, it's gonna move some of the metrics more. All right, I think I missed one somewhere. The other one to look at is a lagging indicator and leading indicator. Now, lagging indicators, whatever is done and dusted with, for example, your sales sort of churn rate. Now, once the customers are gone and you're tracking your churn rate, I mean, there's really nothing you can do about whatever is gone. You can always take a measure in future. But leading indicators typically would show you, this is what it is based on this, you are able to predict the next set of things. For example, sales funnel. So if you really keep a track on your sales funnel, it's gonna tell you how much are actually gonna convert. Now, if this is going way too thin, therefore, obviously, your sales will go down. So you have to keep a balance between what are your leading indicators and know about them and lagging indicators. If you don't know, then your interpretation will be always wrong on that sense. So, we quickly run through some framework and one specific that I'm gonna go deeper into. There are multiple sort of frameworks. So now what metrics do you track is based on the stage that you are in and there are multiple frameworks to figure out which stage you are in and what business you are in. These two factors determine what metrics you track and we're gonna talk about it. So this is Eric Ries' Lean Startup book, if you've read. It's three engines of growth, as he said. One is a stickiness engine, the other is virality engine, and obviously the price. Now, stickiness engine, your whole key thing is to sort of, one metric that you have to figure out is churn. So you have to acquire customers at a higher pace then you are able to lose out them. That would mean that what some customers are still sticky and if you keep doing more, you are gonna stay longer in the business. Now, once you have a sticky base, you have to figure out how do I make it to the next level? So how do I make it go viral and do more of some of the growth hacks so that more and more users start coming in? And then you have to go on the price which is when you start making some revenue. I'm gonna talk about, and this is again the Lean Analytics stage. So, see more or less I'm gonna draw a comparison here. More or less it's simple, that you first figure out your initial set of customers, make them really, really sticky. And Amy Jo also, Amy also talked about that in the game thinking. Once it is really sticky and you figure out that people are really using it, then you figure out how can we make it viral, do more of it, how do they refer other friends and family and stuff to use it. And once you're there, then you start figuring out how to monetize that part and then scale and do it on a more consistent basis. So this is one particular framework. So this is again, this is one more framework which people use. So in Lean Canvas you figure out, I'm not going too deeper. We figure out what are your problems, what are your solutions. And you figure out your key metrics that you always look at it and always change your behavior based on some of this. This one is what I have applied and I'm gonna go deeper into it. It's really, really simple. I mean, I think if there's one takeaway you can take from the session is Dave McLeo's Pirate Metrics. Really helpful to understand any business. And I'll tell you how you can try that. So basically buckets into five blocks and in our product we were actually done our roadmap based on these blocks and therefore KPIs would also do that. So the first thing is you have a product. Now you have to figure out how do I acquire users? So that's your acquisition track. So now that could retain from how do you acquire to how many are you acquiring? So you can figure out a bunch of KPIs around that. How are you acquiring it? What channels are working for us? A whole bunch of things will come under acquisition. So there the KPIs could be unique page views. You're tracking it. Number of users you are getting it on a week on week basis. Number of downloads you are getting it if you're in an app business. All this is your acquisition. So you really have to figure out how are we acquiring. Now in a very, very early stage you may not even bother too much because you are really working with a very small set of users. And you're sort of in a concierge mode or wherever you're working with them. So therefore you know, I mean I have a bunch of hundred users that I'm gonna go back to. So this may not matter, but this will become really useful once you start scaling it. The second part is activation. Now that is what is that, is your conversion. So now once you, they come to your website or they come to your product, did they convert or not? Did they do a particular thing that you wanted? Did they sign up for example? If you're doing it, so that sign up is telling you that you know what now you've activated the user. First time he has used it. Now this is also similar to onboarding in game thing, but it may not be. It's more of conversion in Ateesan or Palin. Now, and that activation also talks about the first time experience of a user. Now once you have acquired them, are they coming back? That's your retention. Most of the startup would struggle right here. This is where your product market fit also is that you have got, you've spent a millions in getting the app downloads, but if people are not sticking to it, not using it in a frequency that you would want to, it's complete waste. So retention is the most key aspect right there. And 70% of the product development also would happen for retention. If you have customers, you're building bunch of features you're building because you want to retain them. I mean obviously revenue is one part, but also you want to retain them. I mean there are like half of the time people will say if you don't build, I'm going. So you're struggling on the retention side. Then you have your revenue, where is your money coming from, and the whole diploma. If you just, I'm gonna go through an example, but this is really really powerful way to sort of categorize what you're doing in your entire startup. And you know at the counter we've done implemented this. We also implemented our whole roadmap. So now you pick a metric and you figure out what do I need to do to improve this. And you have your backlog, right? And then you can rank order, prioritize. Now in every spring you can figure out which of this are we trying to improve. Are we trying to improve retention? Are we trying to, you know? And we put this whole structure where our development team, we're talking to product managers, you first you tell me which of you are you picking this? Are you picking a retention? Oh, okay, this is what we need to improve. This becomes your sprint theme, in so to say, that we are working on retention. In retention, what KPI are you trying to improve? That's the next part. And I'm gonna come to you. And that's where you start the whole language changes. Now product managers also are forced to think, okay, what exactly am I trying to solve? And they have obviously hundreds of features, but this framework will give you that whole structure to really think what do I need to do? And you will also look at your metrics. If you have set up metric will help you figure out, okay, my retention is going poor. Now I need to do, what do I need to do to improve it? And then people can brainstorm, figure out what they need to do, come up with the hypothesis and validate it too. Yeah, so it's called R, if you read, it's A-A-R, R is the word, you know? R, you know? Yeah. So the other thing this also does is, it makes a very, very good funnel. So if you have, you know, sort of, and you know, this is a sample example, you know, I picked up from somewhere, if you have 1000, if you have 100 customers visiting your website, right? And you only 30% signed up, you know? So you certainly have out of 130% sort of activated, out of them actually next week, only 3% came. So your retention rate is three, out of them only one sort of, you know, converted into a paid customer or maybe you give you some money. So those are like 2% and then the amount, they give you money and then how many referred you is only 1%. So if you, on a every week basis, you were able to draw this funnel and see how are you doing, you know? You might be acquiring, you know? So this you can apply it on a very scale at a growth level also and at a very, very early stage also. It sort of sort of fits across all of this. I'll just show you what we did. So now this is a framework, right? I mean, you can't just completely copy it. What you really can do is take away, you know, the gist and what we did, I mean, it's a personal thing that we're sharing. So we figured out that for us acquisition, from a marketing perspective and a signer perspective is different. So, you know, I made a sort of extension to this and added a one more A at the first, which is attention. So how many actually are we drawing it to our website? So we sort of started to bucket that totally different because that was a, you know, metric for marketing and not product per se, right? I mean, product was doing, you know, more from once you signed up, what's your experience is what we were trying to, you know, break up. So we gave marketing a bunch of metric and say, you know what? And we really listed on all the questions that we want to be answered, right? One of the things about metrics is that before even you go to, you know, X and Y ratios, you really have to just think, what question do we want, you know, answered during the whole startup journey? So we wanted to figure out that, okay, we have a product. Now let's look at how many people are showing interest in us. Anything that we do, and you know, how do we measure it by looking at unique visitors on our website? I mean, you know, is there any other way? No, I mean, you know, that's the only thing, you know, if you do anything, it has to come somewhere as a unique visitor and we will be able to track whether that is improving or not on a week-on-week basis or not. Now we want, the second thing we want to figure out is, okay, if people are finding us, where are they finding us? How are they finding us? What channel? So, you know, we listed out, you know, that there are like the real estate website or it's just simply organic or it's like directed. Some of these ignore, which was our internal lingo. Now, once they, you know, sort of coming to our website, are they, is the content good enough? What, so how do we measure that our content is really good enough? Because when you're, you know, one of the thing that you validate your, sort of value proposition is, users came to your website, are they understanding that, you know what, this is what I was looking for and it's there? How do you know that, right? So we wanted to ask a question based on our, based on that, we can copy our, you know, sort of tweak our copy on the website, right? So we figured out what will be a good KPI to indicate that. So that's typically your average session time on the website. If it's very low, obviously people are not going to make out what the hell they came for and they are exiting. So your exit rates will, you know, sort of include. So there is some correlation between average time spent on the website and, you know, exit rate. Similarly, then we started, okay, once we have all of this, let's figure out how many, you know, people are signing up, how many users are we acquiring? I mean, some of those are internal, you know, metrics, I've removed the data and do. How many, you know, mobile apps, you know, mobile app downloads are we getting on an Android, iOS? So you have to do an acquisition at a different levels. And then you start tracking it on a week and week basis. All right, so I want to do a quick exercise if you guys want. So any questions so far on, you know, Dave McLeod thing? Because really, this is a really, really powerful framework that you can apply on anything. You know, even if you can apply on the whole big enterprise sort of development if you're doing. If at all it is more, you know, business outcome driven, you can apply that framework right there. Any questions here? No. I want to figure out, so if you've seen ConFingen, right? Just want to take an example, you know, we in directions build this whole product, ConFingen. If you really have to track one metric to figure out, is he acquiring enough, you know, sort of in terms of acquisition, what could be that metric? Make a guess, I mean, let's test it out. No one knows that, you know, you have to just figure out. So it could also be both, right? I mean, we could always choose to make one and this will come up. This will come up based on the question. So now we want to track both probably. Probably if you track both, you might figure out the number of conferences is correlated with the number of users people come, right? Yeah, I mean, there could be correlation. And you can figure out only after tracking it. So if you've seen all this, we're tracking all of this. It's not that we are sort of making decisions based on that. Decisions get based on a bunch of other things that I'm gonna talk about the whole framework. But this is the data. You can always refer back and the more you're digging into the whole session, you'll figure out some causal relationship between same and then suddenly that'll give you a Eureka, okay, this is what we should be doing. All right, now once, sure, absolutely. They do, they do, they do. So that's why I'm gonna come to one metric that matters. So, you know, and that's what out of the whole laundry list, you really have to pick out just one, just one that you're really gonna focus on. Now, one of the challenge with that is that it's very, very hard to come up with that one thing, right? So, and also, and that keeps changing. And it has to change based on the startup journey. It could even change based on the sprint, as I said. See, for us, you know, also we came with this challenge, but what we were able to do is we were able to create four micro teams, each with their own metric that really matters to them based on the customer personas. So we figured out that, you know what, we are really dealing with four different customers. It was an enterprise sort of product. We said, you know what, this is one persona and this is one team, you focus on this particular metric. This is your holy grail. You make decisions based on this. You make roadmap based on this. You make your releases based on that. So, that's how it could help. All right, so there are multiple frameworks that we just talked about. This is where all of them converge. It's like, this is Ash Moria's lean canvas which talks about problem, customer segment, unique value proposition. I mean, you can do it for some of this. Then there is a lean startup framework which says problem validation, solution validation. So once you have a problem, you figure out a problem, validate if there is a genuine problem or not. Once you have a solution in mind, you figure out is this solution gonna work. Make a MVP, learn from it, go through it. Now that same thing sort of maps it to a lean analytics, which says empathy. Empathy is the first stage which basically does your product market fit and sort of cuts across. It says that you have to find a problem that you can really solve. And once you've figured out a poorly met need that can reach to a sort of market segment, that's where you have crossed the empathy stage. Once you've figured out how to solve the problem that people are really, really using it and getting benefit from is your stickiness. Now that stickiness is more of retention in Dave McLeod. So more of these, this sort of draw parallel. So you have to really pick one that is more easy for you to use and the whole team can sort of relate to. You picked up Dave McLeod and really worked for us. I mean, you guys can take your own thoughts. Then you have virality, revenue and scale. Typically, it basically means you find a problem that is worth solving, figure out a good solution, get a bunch of 100,000 customers out there, validate it, build a MVP. You can validate it through multiple tools and hacks that are available. Once you have it, you start figuring out that are they really coming back? So your stickiness engine is building or your retention is improving. You have to reach a stage where you're really, really sure. Now, I think we are able to retain. If you pump in more at the top of the funnel, you will be able to retain some. Now, if you don't have a retention, basically, the more you're acquiring, you're just losing all of them. So it'll always be a leaky bucket. So you try and fix that leaky bucket first before you actually go more on scale. That's the whole of premise. It's all of a checkpoint, more of a mental checkpoint. And one of the funny thing is that if you ask a startup which stage they are in, they'll always think that they're on the next stage. Okay, no problem, you know, product market we've already done with, you know? But actually, they're sort of struggling. Also, there's a lot of overlap at times, right? So for example, some are in a pressure to figure out their revenue engine right at the early stages, right? So they're iterating bunch of this together. And that's where it sort of gets, you know, right, right, right, right. So that depends. So the whole parameter depends on two things. One is which stage you are here and which business are you in. So they're like, they've figured out six business models. Sure. So they're not stages, they have a different approach that, you know, you, so a lean canvas, if you look, right? Right, you can, right, you can. And that's where it's somewhere in the revenue part, right? It maps to the revenue. So they have a different approach. For example, if you've read Ash Maria, Ash Maria's book. So he says, this is your canvas. These are your problems, this is your solution. You keep iterating based on the hypothesis and keep changing these. So this basically, you know, if you have a solution, you're building a solution, you build a MVP, learn from it. And ultimately you sort of, you know, if it is not working, you again change the solution. So you're iterating right here. So you were, that means you're still in the stickiness business, right? So that same stage. Now, if you've figured out a revenue stream, a lot of startups don't have a revenue stream. They only have a hypothesis that, you know what, if I reach a stage, I'm going to start making money. Which means they have not reached yet. Now, if they are reached yet, then obviously then they are, you know, doing something on the revenue side out. So it's, it's not a very, very hard comparison, but you know, it sort of relates where each of these models converge somewhat. And you can pick one, and that should be fine with you. I mean, and I'm going to, you know, talk about the whole analytic, you know, sort of metric cycle that you can go through by picking any of these models. I mean, based on, you know, at the end, I mean, it's your own startup journey that you're going through. The basics remain the same. That you want to solve a problem. You want to iterate very, very fast. You want to go through the build, measure, learn, loop very, very fast. And you know, sort of keep learning from it and input. Yeah, so I was saying the whole metric that you track depends upon two things, you know, your stage of growth and your business model. Now in business model, you have e-commerce model, which is your typical flip cart and, you know, sorry, not flip cart. I mean, some of the sort of portals that sell. So flip cart sort of comes on on a, yeah, two-sided marketplace also. For example, we were in a two-sided marketplace. So we had to look at the server side, buyer side. So some of the metric could be different. There is a SaaS, which talks about more of churn and customer lifetime value, you know, though some of the metrics you can reuse in most of the other places. And then you have user-generated content. Let me just show you. So I have a graph which talks about, you know, one metric across all, and we just go to that. Now, once you've figured out which model you are, and a lot of time these models will overlap, you might be a mobile SaaS company, for that matter. So there's a lot of overlap that will happen. In fact, if you read a book, which I have a recommendation, that has a clear flow chart for each of these models. So you're able to relate exactly how do we go about, you know, in doing that. All right, so we've figured out, you know, what type of metric to look at, you know, what are the different ways, you know, by which you should look at the metric, what stage you are in, what business model you are in. Now it's a simple flow, like, build, measure, learn is more of on the development side. This is more on the hypothesis testing and analytics side. You pick a KPI, what we call it is, you know, draw a line in sand. So if you figure out I'm in a retention stage, you figure out, all right, what should be our retention? What's the ideal retention in a business like ours? You know, it's like 10%. It doesn't matter if it's six or seven, you draw a line that, you know, what we want to reach 10%. Now once you have it, you have to figure out what are the potential improvement that we can do, or your backlog, to reach 10%, right? Once you have it, you prioritize, if you have a data to back, okay, this is how we're going to do it, it's fine. Form a hypothesis, design a whole test around it and we saw the whole hypothesis testing loop. Pretty much it applies right here. Figure out, measure the results. If it improved, your metrics have improved, you're tracking it on a week on week, month on month basis. If it is not, obviously you have to pivot or give up. If it is fine, obviously you've improved something, learn from it, probably change the, you know, sort of the benchmark or line in sand. It sort of gives you a quick flow chart that you can run through. The way we did it, more of on the sprint side. So we figured out a sprint theme that we want to improve retention now. It could happen and one of the struggle that we always had is that we figured out, okay, we're doing this for retention. So we want to improve our churn rate, for example, right? So a lot of customers are giving up. So we figured out our hypothesis is that people are going out because of these five things. So we need to do, we put it in the sprint backlog and say, if we do all this, our churn will start improving. This might happen that the churn actually started improving for sprint down the line. Because by the time it hits the release, people start using it, you have, you have to wait for a while to make it. But it gives you a good structure that at, and we used to have a Kanban board where we put, you know, this is the current hypothesis in, you know, in testing. And these are the KPIs we are watching for. So in every sprint meeting, people come back, okay, tell me, you know, what did we improve on the last one or not, you know? So you go back, reflect on it, and you know, if there is more thing on the backlog just to continue that, you know, start doing it. All right, so this is where we're talking about the one metric that matters. So basically based on the whole stage you are in and the business you are in, you have to really figure out what is one metric that really, really will make or break your whole startup. The closer you are to figuring that out, the better are, you know, your own iteration loop, right? So there's a, you know, why one? Obviously, I mean, you know, you guys also said, I mean, that one will let you focus really, really really sharp right on that goal. It'll, you know, sort of also help you to make a, you know, sort of draw a line in the sand that, you know what, this is a metric and you know, we need to reach here. Now that here could be off, but at least you have a stab at it. Focus the entire team right there because everyone is now talking the same language. We are trying to improve churn or we are trying to improve metric. A lot of ideas will come from the team since they know that this is what you're trying to improve, right? You wanna point, if you're improving 50 other things, people won't even remember, they won't even contribute to it, you know, to that. And it will sort of help you to build a whole experimental culture. Kalpai's give you a talk which is like a, you know, making the whole experimentation culture which is really, really great, you know, in Charlotte. So there's a quick chart which is also available right there. So if you are in an e-commerce business and trying to figure out stickiness, what you really care is the loyalty and conversion, right? And you know, they could also pick one conversion, for example. I mean, if people are coming to my e-commerce site, not buying anything, it's like bullshit. Nothing happens. So you really focus on conversion right there. Now, if you already having conversion, you reach the stage, the second level of stickiness is, you know what? How many times people are buying it, right? So I have a big basket grocery business. If people are only buying groceries once in a lifetime, it doesn't help me. I want them to buy. So you draw a line and sign that I would really want people, my customers to buy twice a month, at least. So you draw a line and sign and then you figure out, okay, what do I need then? You know, they start instrumenting your thing. How many customers are not buying twice a month? So what is going wrong there? And you start, you know, figuring out to probe them by push notifications. So that's why it's a line in sand. You clearly will, you know, absolute, absolute. And you know, it goes back to this loop, right? So if, you know, yeah, you draw a new line, right? I mean, right there. I mean, you try again, you draw a new line. Yeah, and that will happen because a lot of time you won't find the benchmark in the industry. Yeah, okay. All right, so this is a quick, I think this chart is available on the book and you'll be able to figure it out. Based on that business, for example, we, you know, we focus a lot on retention and we defined our own metric. I mean, we wanted to figure out that, you know, on a social network, for example, if I, you know, we were building a bunch of social network, we wanted to figure out that we want at least people to do three sort of discussions in a month. Without that, it was a really, really hard benchmark. I mean, you know, the moment we did, our whole filter went really down and we were totally fine with that. We got a bind and said, you know what, we're taking a really hard call. But if we are really able to improve that and reach that level, we are really something. We are onto something, otherwise not. It turned out that after three months, we could only get to two, which is okay. I mean, then we said, okay, two is the benchmark. That's the best we could do. If we improve something, it's fine. So we drew a line in the sand and then worked towards it and, you know, sort of, that's how we could, you know, save people. All right, so quick summary. You know, it, so build your whole customer life cycle, identify where you can, you know, sort of leverage, you know, in what stage you are. Try and create, you know, you sort of state your goals in a very measurable way, something which is measurable, understandable, you know, in ratios. Pick one metric for your startup that really matters. That could change over a period of time. It could change on a monthly basis. It doesn't matter. You could also take a stab at multiple metrics on each sprint. I mean, you know, one metric on one sprint and, you know, over a period of time. Within the same stage, you know, if you're on a retention stage, see, one of the challenges that we faced is that if you focus too much on retention in one shot, beyond a point, I think we felt that, you know, we're just, we're just spreading two things. We're doing analysis paralysis. I mean, you know, what, what it is, it is. Let's figure out that probably putting more people on the funnel might improve it. Right? So, so that, we then, we had to, you know, we changed gear to, you know, acquisition for one or two months. So one sprint, we got, okay, we got good, you know, this is the persona. These guys, if we acquire, we'll come up. So we started move back on acquisition, you know, at some time. So, and it's a lot of it is experimentation, what we figure out, what hypothesis that you, you know, come up with and sort of try to improve. Run your experiment, test, optimize, figure out your own hacks, you know, if you're able to find a causal analysis between something, it's really great, you know, that will just help you, you know, in a more better hack. Pick a new metric and sort of keep that. This is a really good book, you know, most of the concepts are from here, you know, and it talks about in a very, very simple language. There are a bunch of blogs. I'm gonna just add one or two references and you know, share that slide out. All right. Thank you. Any questions? How many of you are doing some sort of analytics or some metrics right there in New York? Did this framework help? You know, it's very, very simple. And that Dave McClure framework is really, really powerful. I mean, trust me, I mean, the more I've explained it to people, even executives, they'll just get that. No, okay, this is a five bucket and you really tell them that, you know, what we're trying to prove this here. So there are a bunch of things that depends upon your business. Now, if you're in an e-commerce business, you will be able to figure out industry benchmarks or, you know, talk to a bunch of people and say, this is what it is, right? I mean, be on a point, there's no... Little bit. I mean, see the whole thing is, there's one 5% rule also. In really, really good startups, they're improving their metrics 5% a week, most of these metrics, right? So especially acquisition and retention, they're like, you know, Slack. You go look back, Slack data, they say, you know what, 5% you have to hit that number. You can always figure out that, you know, what in my kind of thing, you know, it's a discovery problem. So we'll do it at this stage. But at a growth, you really have to hit upon that. Next time, right? Great. No, and there are a bunch of growth also. I did not add tools, but there are a lot of tools and hacks to do. For example, the simplest one, you know, you could do, you know, if you're in a more customer enterprise, you do NPS and you're figuring out, okay, NPS is what we really want to improve, right? Net promoter score. Basic app could be an app, you know, you launch an app, you give them a pop-up. Hey, did you like the experience? If not, the no takes him to a quick feedback pop-up, which you start learning, you know, why something went wrong, right? You know, so you're talking to your early, very, very early customers. So depending upon your stage of growth, as I said, you have to figure out your own hack and the metrics that would really improve, you know. And that's how you will be able to validate the whole thing. I mean, if you're in an app business, you really have to work, will I, you know, so what do I want to achieve out of the app? Do I want 100,000 downloads? If that, you know, is that the, you know, sort of assumption on which you will make revenue, then obviously you have to do something about acquisition. If not, then it is obviously one more feature where you can track it, what people are using it for. And that could give you a picture of where to improve or optimize. All right, thank you. Thank you so much.