 All right, so welcome everyone to this webinar. I really hope that the next 30 minutes are of good value to all of you in terms of learning. So today, we're gonna see how to break down a complex business problem using data and user research and then how to use those insights to build some impactful products, all right? Okay, but then before diving into that, a bit of introduction about me. My name is Lokesh, I'm the scared guy that you see on the right side. In terms of my career, I have mostly worked in tech. I started working briefly as a developer, then I moved into data analytics and post which I moved into product. A few companies I've worked in the past are at Oracle, Meta, Swiggy, which is a food delivery company in India. And currently I'm working as a group product manager at Glovo, Glovo is a Spanish food delivery and quick commerce company. It operates in 25 countries across Europe, Africa and Central Asia. And also in case you want, you have my LinkedIn and Twitter profiles here. Cool, okay, so now that I'm done with the personal branding, just a quick disclaimer, the case that I'm presenting today, although it will give you some very real learnings, all the data, all the problems that you see are dummy problems and dummy data. The intent here is not for you to become a master in the industry that we'll be discussing today, but rather learn some techniques on how to break down a complex problem. So it does not really matter how accurate the data actually is. Okay, so then let's dive right in. So let's talk about a fictional company called Jimmy. So what is Jimmy? So Jimmy is nothing, it's a monthly subscription program where you can pay 79 euros a month and then get access to every gym that is registered with Jimmy, right? That's quite an interesting value proposition. And in case you're wondering how you can become a Jimmy member, you'll have to move to a country called Jimmy Land. And Jimmy pretty much operates in all the big cities, the five big cities of Jimmy Land and it does not have any competition in Jimmy Land. It is a sole player in its industry. And now that we know about Jimmy, just diving a bit deeper into Jimmy's business, so Jimmy has 50,000 monthly active users and Jimmy does about 600,000 check-ins across all of its gyms in a month, right? So if you do like a simple math of dividing the check-ins by the monthly active users, you see that an average Jimmy user does about 12 check-ins per month. Okay, and then also purely in terms of selection, Jimmy seems to have a very solid selection, okay? So we see that it has 100 plus gym and fitness brands and in total it has about 7,500 gyms, studios and individual trainers across the country. So a very, very solid selection there. All right, so Jimmy is doing well. It has a good selection. Then I'll be discussing this case, right? So then what happens is that one fine morning, the CEO of Jimmy writes a letter to the senior management and here you have a very quick summary of what the letter basically said and I will read this out for you all. So the CEO is saying that we want to expand into more countries and we also want to get into the B2B business line with portfolios. And for this, we need heavy investment in marketing and sales. We have been struggling to get funding. We need to make more money in our existing markets and funnel that money into new areas but we have no money to invest in user acquisition. Okay, so then let's break this down. What exactly does she want to say? So this is a very common problem faced by many early state startups. So they're doing well with a small power user base. They probably are break even or even profitable for that matter but there is no money to acquire more users and expand into new geographies, right? So then the problem essentially is that Jimmy needs more cash in the bank, right? And then without going into all the complex marketing and finance jargons such as gross margins, contribution margins, in very simple language what Jimmy wants to do is increase the money in the bank, which is nothing but Jimmy's revenues minus Jimmy's costs. Again, this is a super oversimplified explanation but for this exercise, let's stick with it. So we want to increase the revenue and decrease the costs and how do we do it? Okay, so then let's first start understanding what exactly constitutes Jimmy's revenue and Jimmy's costs. Okay, so then Jimmy makes his revenue through subscriptions, okay? Hence the revenue is nothing but the monthly active users multiplied by the subscription cost and we saw earlier that the subscription cost is 79 euros, okay? So then the revenue is nothing but the 50,000 monthly active users multiplied by the subscription cost of 79 multiplied by 12, which is the number of months in a year this gives us an annual revenue of 50 million euros. That's pretty solid for an early state startup like Jimmy, but revenue does not give us a full picture. We also have several costs that are involved, right? So the way Jimmy's business model works is that every time a Jimmy user goes into a gym and does a check-in, Jimmy has to pay a fixed, pre-discussed check-in fee to the gym, okay? Hence in this case, a very simple math equation for the cost is nothing but the monthly active user multiplied by the average number of check-ins that this user does every month, multiplied by the average check-in fee that is pre-decided with the gyms, okay? So this is the cost equation. Okay, first let's deep dive into the revenues. How can we increase the revenues? Sorry, so there are two ways in which we can increase the revenues. The first way is to increase the monthly active users, okay? And this can be done in two ways. The first way to increase your monthly active users is to actually acquire and activate new users. Now we can have different definitions for acquiring and activation, but in simple terms, acquiring is making a user pay on Jimmy, which is basically to become a subscriber, and then activate probably is to make the user have a certain amount of engagement on Jimmy. So the first way is to acquire and activate new users. The second way, which is more interesting, is to retain existing users and to reduce the churn. And there are many ways of doing this, such as gamification, building loyalty programs, I'm not going to get into specifics of these, but you get the idea, right? But there's a catch, and the catch is that we cannot really acquire and activate new users because like you see earlier mentioned, there is not enough marketing money that Jimmy has today. So the first option is, is not really a feasible one, okay? The second way, with which we can increase the revenue, remember the equation of revenue is subscription cost multiplied by the monthly active users. So the second way to increase the revenue actually is to increase the subscription revenue itself. And like two ways that come to mind, the first one is to hike the prices. So the current price, as we know, is 79 euros. If we increase that 79 to say a 99, this will naturally increase the subscription revenue, right? And the second way is to create some tiered subscription plans based on benefits. For example, a basic plan, which is 60 euros, a more intermediate plan, which is 80 euros, and an advanced one, which is say 100 euros, right? But the first option, which is to hike the prices is not a recommended option. Because think about it, if users get a notification that from the next month they will be charged a higher amount, many of them may naturally churn and this will actually hurt the monthly active users. Cool. Okay, now we are done with the revenue side of things. On the cost, we see the cost is nothing but a function of average check-ins per month and the average check-in fee. So then how do we decrease the cost? Purely based on math, some of you may say that if we reduce the average number of check-ins a user does every month, we can reduce the cost. But think about this. If we reduce the frequency of the user, they will go to the gyms less often and in turn what will happen is they will start seeing a lesser value in Jimmy, right? And in the long-term, this will hurt retention. So while in the short-term, purely based on math, this may seem like a good option to reduce the cost, but in the long-term, this will have a very massive impact on the growth or the monthly active users of Jimmy. So this is not a recommended option. We should not build products or we should not have strategies to reduce the monthly check-ins. Rather, another way to reduce the cost is how can we decrease the check-in fee itself that we pay to the gyms? The first and the most simple way to do this is asking the gyms to reduce the fee. But again, this is something that comes with the caveat that if the gyms have to get a lower amount of fee for every check-in, they may see the revenues dropping and they may start to leave the platform and then this may hurt the selection itself on Jimmy, right? But then, is there something else that we can do apart from simply reducing the money that you are paying to the gyms? So then imagine that you are the PM responsible to decrease the check-in fee that Jimmy pays to the gym. The first thing that you would probably want to do is look at the average check-in fee that Jimmy is paying today. So how do we compute that? We look at all the 50,000 monthly users and see that for every check-in that they make, how much is the average check-in fee that Jimmy is paying to the gyms? Imagine in this case, it comes to about five euros and 25 cents, right? Pretty neat. To put this into perspective, think about this. We saw that Jimmy does about 600,000 check-ins every month, right? At an average, for each check-in, we have to pay five euros and 25 cents to the gyms, right? For 12 months, this would come around 40 million euros a year. That's the total cost that Jimmy is bearing by simply paying the check-in fee to the gyms. And if you remember, we computed the revenue was about 50 million euros. So this leaves Ziggy with 10 million euros in the bank after reducing the cost, that is 40 million euros, right? And the problem now is how can we increase this 10 million euro, the money in the bank for Jimmy after reducing all the cost by decreasing the average check-in fee that Jimmy is paying to the gyms, right? Okay, okay. So, sorry. So now let's deep dive further. So averages normally are never the answer because averages can be super skewed by outliers. And in this case, although we see that the average check-in fee is five euros and 25 cents, but it's not that every check-in costs the same amount to Jimmy. Some gyms charge a higher fee, some gyms charge a lower fee. Hence, it's very important for us to actually have a look at the distribution. And if we see the distribution here, we see that almost 90% of the gyms pay Jimmy a commission of three euros, right? Sorry, they charge Jimmy a commission of three euros. Then why is the average check-in fee so high? If 90% of the gyms are actually charging Jimmy only three euros in commission, then why is the average check-in fee that Jimmy is paying so high? For that, maybe a better analysis to look at is how much is the average check-in fee that Jimmy is paying at a check-in level? And we see the trend reverses here, right? We see that a high number of check-ins that Jimmy is doing costs Jimmy six euros. Now take a moment and see what's happening. Most gyms are charging Jimmy only three euros per check-in. However, most check-ins actually are costing Jimmy six euros. So how is that even possible? And the answer is this. We see that 10 big gym brands with 100 gyms contribute to 60% of Jimmy's check-ins. Now, these big brands charge a high check-in fee of six euros to Jimmy, right? And this is Pareto in real life, right? You see that 100 big gym brands charge Jimmy a six euro check-in fee and then dominate the check-ins on Jimmy. 60% of check-ins come from just these 100 gyms. And while the remaining 6400 gyms that charge Jimmy a very low fee get only a 10% contribution to the check-ins. So it's purely because of this variability in the check-ins with most of the check-ins going to these 100 gyms that Jimmy actually has to pay a much higher check-in fee, right? Okay. So then how exactly did Jimmy land here? So the first question that's worth deep diving into is why do these 10 brands have such a high check-in fee and is it good for Jimmy's business? So firstly, due to a large number of check-ins and the brand power that these gyms have, these gyms get a very high negotiating power. So they can actually negotiate and charge a higher check-in fee to Jimmy. Consequently, these brands then become very dangerous for Jimmy, right? Because not only do they charge a very high check-in fee, but they also control a large amount of business at Jimmy. So tomorrow imagine if any of these gyms break the partnership with Jimmy, Jimmy will actually lose many customers, right? So it's better for Jimmy to not have a lot of their check-ins concentrated to just these 100 gyms, but rather to diversify the check-ins that they get across all the selection that they have. Okay. So we looked at the lens from the gym lens here, right? But also it's important to look at the user lens. So does every user on Jimmy make 60% of their check-ins from these big brands, right? And what we can do here is segment our users based on their lifetime orders and their monthly frequency and see their behavior, right? So then let's deep dive into this. And by doing so, you may end up with a table like this. So let's see what do we have here. So we see that users with less than or equal to five lifetime check-ins have 85% of their check-ins which is almost all of their check-ins coming from these top brands, okay? And honestly, this is expected, right? Because big brands are generally trust builders for new users, right? And also most of Jimmy's acquisitions is happening through big brands also partly because a lot of offline and performance marketing is done with these big brands and Jimmy in collaboration, right? So that's the reason why most of the new users or the early users are mostly checking in at big brands. However, even for the repeat users who are the users with more than five lifetime check-ins, sorry, we see that almost half of their check-ins are coming from these big brands which is not as much as the new users but it is still a very significant number, right? So then let's further break this down. Let's look at the repeat users. So we get a very interesting chart here as you can see that as the monthly frequency of a repeat user increases, we see that the share of their check-ins from the big brands also decreases, right? So we see that a user who makes four to five check-ins in a month does 70% of them from big brands. While a user, say for example, with 13 check-ins in a month which is the second last bar that you see on the graph, is only doing 25% of their check-ins from these big brands. Very interesting and this gets us to a very interesting hypothesis which is that as your check-in frequency increases, users in general tend to get more experimental and they are willing to try the gyms outside of the top big brands. However, for the low and medium frequency users, most of the check-ins still happen at the big brand gyms, right? So we looked at a lot of data. I know these are a lot of numbers to consume, but then let's look at one more data point and then we will summarize all of these things, right? So we see that 82% of a user's check-ins come from the same gym and in most cases, this is a big brand, right? Okay, now let's make sense of all the data that we've seen so far. Let's move to the next slide. All right, so what do we have here? So we see, number one, that we acquire and activate most of our users through big brands, okay? And then users form these early habits with these big brands and hence we see a lot of low and medium frequency users. They keep going to these big brand gyms again and again all the time. These early habits then form very, very strong repeat behaviors and as a result, you see that 82% of a user's check-in check-ins happen from the same gym, okay? And then this as a whole leads to a very high check-in fee in general for the entire platform. I have one more slide to summarize everything. So let's go. The first one, big brands are very important for new users, right? To activate the new users, we need these big brands. These are trust builders. Number two, high frequency users already have a low dependency on the top brand. So this is not really a problematic segment for us. This leaves us with repeat users with low and medium frequency as a segment on which we can have a very high meaningful impact. Also, the problem is arising because users get acquired with big brands and they keep going back to them as a result of very strong repeat behavior is formed. This increases the dependency on big brands and herds the profits of Jimmy, okay? Cool. Now that we have fleshed out the problem, let's look at our customer application. How exactly are the customers discovering these gyms, okay? So we have three discovery avenues on Jimmy and let's look at all of these, okay? The first one that we have is the gym listing, okay? Gym listing is nothing but think of it as any e-commerce application where you have the entire catalog listed. So this is the gym listing which has all the gyms registered with Jimmy listed and probably the kind of numbers you will see here is that 55% of discovery actually happens on the gym listing and 74% of all the reservations that are coming from the gym listing page are listing, sorry, are reservations from the top brands. So to put it in other words, of all the reservations that are happening from this page, almost three quarters of those are from the top brands, okay? Another way users would discover activities and gyms is from the activity discovery page which is nothing but shows you all the activities that are being offered on Jimmy. For example, you see here cardio, Pilates, Zumba, et cetera. And typically you would see about a quarter of discovery happening from this page and around 30% of the reservations from this page will be top brand reservations. So this page is actually better in terms of discovery of non-top brands compared to the gym listing, right? And yeah, then the last page or the last avenue from which users can discover gyms is actually the search, right? And typically in any e-commerce you would see about 30% discovery happening from search and typically about 38 to 40% of the reservations happening from this page could be from the top brands, okay? Cool, so next let's now jump on to the user research. So now that you have all the data, it's also very important to qualitatively understand why users check in from just a few top brands and some insights which you could potentially get by talking to a few users are number one, there's generally an element of mistrust on unknown gym brands. So users are not very sure if they will get good equipment, if they will get good training, with these lesser known brands. The other insight that you could potentially get is that gym training is generally a very, is a group activity, right? For a lot of professionals, for a lot of friends, they want to go to the gym together. And generally in this group dynamics, the popular choices are the ones that normally win. Thirdly, there's also an element of familiarity and routine that comes if a user goes to the same gym every single day and that's why you see such a heavy repeat behavior among all the users. And lastly, you can probably hear from users that due to very heavy offline marketing, both from these top brands and also given how Jimmy generally acquires users through these big brands, these big brands then become the top of mind for many users, right? Okay, so now that we have fleshed the problem from both a data and from a qualitative perspective, let us see how we can define a good metric that we want to optimize for. Sorry. Okay, so a few key points before we choose a metric. The first one is a good metric as a PM. The metric that you should choose should not just focus on business value or business outcome, but rather on a user value, right? What I mean by this. So we know for a fact that our high-checking fees are actually a result of users repeatedly checking in into top 10 brands. But if we purely optimize for reducing the check-ins in these top 10 brands, it's a very poor customer metric because we will end up deboosting all of these brands from everywhere, creating a very poor user experience, right? Hence, the primary problem from a user perspective that we really, really want to solve is to basically make users explore more gyms outside of the big brands, right? We know the primary problem that we have is that users, they start their fitness journey on Jimmy with big brands, and then they just hook on to these brands, right? And to solve this, we can either acquire users with smaller brands, which trust me is a very difficult thing to do. Trust building is very important for new users. So the other way, which is probably a better way to solve this problem, is to make users, repeat users, try more activities and gyms every week after they're acquired, right? Okay, so then a good metric. There could be many good metrics, but one good metric could be average number of gyms tried by a user per N check-ins, right? N here could be anything, depending on how long can you run your A-B tests, your experiments, but you get the idea here, right? So we want the user to diversify their gym visits. For example, if N is 10, we want the user to try more and more and more gyms in these 10 check-ins. And this metric, if it increases for us, that is success, okay? Amazing. Now we have the entire analysis. We have a metric to optimize for. Let's now look at the hypothesis on how we can increase this metric. And I could come up with three hypotheses. The first hypothesis, a pretty simple one is, by making users aware of various activities in their city across different fitness, across different fitness centers, we can make them go to a wider variety of gyms, okay? The second one, by increasing trust in lesser known brands, we can demand shape check-ins away from big brands and make users try more gyms, okay? Sorry. And the last one, it's a data derived hypothesis because we saw that the gym listing screen actually contributes to a higher number of big brand orders is that by optimizing discovery on the gym listing screen, we can reduce the dependency on the top brands and make users try more gyms. Amazing. Cool. So the last section then is the potential solutions. And honestly, I want to spend the least amount of time on this section because fleshing out the problem, deriving hypothesis, that's the hard part. Creating solutions, if you have a good hypothesis and if you have good metrics is probably the easiest part. But for the sake of completion, a few things that we can do is, the first one is to boost discovery of activities on the store listing page, right? Now, we know that since most users, they keep repeating the same gym again and again and again, if we make the users aware of different activities that we have on Jimmy and the smaller brand fitness centers that offer these activities, this can be a good nudge for users to go and try these smaller brands, which in turn will increase the average number of gyms that they try in a month. For example, making users aware of, say for example, live Zumba, live dance, live Pilates, et cetera, these kind of activities, a user may move away from their regular go-to gym to a lesser known gym, which is offering some of these activities, right? And one of the user experiences could be the one that you see in the screenshot here. The other one could be like this, on which on the store listing, sorry, on the gym listing page, you show top personal trainers and some trending activities, right? It's a good nudge for the users to go outside of their regular gym brands and try something else. The other one, as I was thinking, since we know that the gym listing page contributes to a high number of check-ins from top brands, one thing we can also do is manipulate, not manipulate, but like build a new ranking for the gym listing. So some machine learning model that optimizes for discovery, for these new, for these repeat users to make them discover new small brand gyms, keeping that optimization metric as a success metric, we can change the ranking of the gym listing itself. That could be another way for us to make users discover new activities and new gyms, right? Okay, another solution, and I really like this one, is having trust markers for small brands. Now, one of the problems which we probably also saw in the user research is that for users, it's far easier to trust a big brand than a smaller brand, right? Which is for multiple reasons, including offline marketing, the only top of mind. So users find it hard to trust new and unknown brands, and how can we solve that? Trust markers on the application, such as ratings and reviews, or some tags, for example, hidden gym, for example, this small gym close to your house is a hidden gym because a lot of people who go to this gym, they repeat this gym again and again, which means that people really find value in this gym, or something like a trending tag, which is in the last one week, this gym has shown a 30% growth in check-ins, right? Or something which is a purely social-proofing-based tag, like 20 plus check-ins in the last 30 minutes, which makes the users feel that hey, okay, since other users are going to this gym, there must be something good in this gym, so even I can try this gym, right? And then from benchmarking, you can see that this is something that Airbnb and Amazon already do. For example, you see tags like Superhost, Rare Find on Airbnb, and on Amazon, a very common trust marker, you see is Amazon's twice, okay. All right, so well, this is pretty much it. So today we saw how can we break down a very complex problem, which in this case was to increase the money in the bank for Jimmy, how can we flesh out this problem? We can break this problem down into small chunks, and then come down to one single metric that we want to optimize for, which in this case was average number of gyms that a user tries in over N number of check-ins, and then how can we build some hypothesis to optimize that metric? And then based on that, how can we come up with some very easy solutions that in the end has a very clear value for the business? Cool, I really hope that you learned something today, and if you have any questions, you have my LinkedIn and Twitter profile listed here, feel free to shoot any questions that you may have. Thank you everyone, and have a lovely day ahead.