 Hi, everybody. My name is Robo Martinez. I'm a product director in King and I'm really excited today to be here with all of the product school community and share a bit my experience on how to go about the finding platform KPS Before I jump into the weeds first. I wanted to go about who am I? My background is in telecommunications engineering and electronics engineering After that I spent some time as a developer very short because I quickly realized it was not what I Like to be doing for the rest of my career. I spent some time later on research in studying ECG signals after that I moved into an auditing company in PWC. I was doing Auditing on mainly financial institutions looking into their systems and processes and how Their statements financial statements were Compliant. So after that, it was kind of like five and a half years ago I joined King started as a business analyst and after some experience there in the player support Area I joined the technology organization as a product owner and then after that I just joined as a product manager and then a product director in the platform team just Looking after what I am doing today, which is the game economy domain and basically we we just provide games with their tools and Technology that enable them to build and operate their their economies What's the agenda for today? So I'll basically be splitting the presentation in four parts first I'll just you know in case you're wondering why should you even care about this I'll give you some arguments on why it is important to do this exercise Later on I'll I'll go through some of the pitfalls that I myself Encountered when when doing this exercise, but also while talking with some colleagues about it and then after that I will just give some High-level hints on how to go about this doing this and what things to consider and finally I'll just give some examples of KP I have worked for me. I'm just trying to make the talk a bit practical for you as well Before we go into the details, I wanted to clarify some terms first The first one is what is a platform? This is something that is quite hype right now. There's a lot of talks about it But basically you have the definition definition here from Evan botcher, which I think it's pretty good So a digital platform is just a foundation Of so service API's to services knowledge and support which are arranged as a compelling internal product And then autonomous delivery teams can make use of the platform to deliver product features at the higher pace With reduced coordination. So basically it's a it's a it's a it's a team or a group of teams that just try to make the the feature teams or the product teams in the company faster Then the second term I wanted to clarify is another one that is is quite Hype as well in the industry, which is what is platform as a product because I discovered what a platform works but right now there's a lot of Literacy around how to deal with a platform as it was a product And we have a quote here from Matthew skeleton That basically highlights that you need to try to treat your platform as a product the same way as you with any other B2b or B2c product And this is something that is really important I'll talk a bit more about that later when I justify why you should even care about platform KPIs, but if you aren't You know, my opinion is that everybody should be treating their platform as a product Because that yields a lot better results than if you're just looking at it as a technical solution And then the final one this at least in my experience is a controversial one but what is the difference between a metric and a KPI and I think that the The word that explains everything is that it's the key there is the key So basically KPIs are your most important metrics in moving your product forward Which means that all KPIs are metrics, but not all metrics are KPIs I think we tend to confuse this a lot and we tend to confuse metrics with KPIs And that's really confusing when you're trying to Sell the idea of defining KPIs or even reporting KPIs to your leadership because When you just have a bunch of metrics and you don't have a way of prioritizing which are the most important ones then There's too much confusion about it Cool, then we go into why should you care? I think the first thing and I kind of briefly touch upon it is that There's a big industry trend and you see here some reference to the technology radar that if you haven't read it I highly recommend it, but in the volume 22, which was from 2020 so it's a bit old One of the key trends that they were highlighting was that you should be applying product management techniques to internal platforms This is the same that Matthew Skeleton was saying but if you actually read online There's a lot of as I said literacy and articles around what are like the huge benefits of doing Taking this type of approach As you know defining metrics and being data-driven is a key thing in product management So if you are trying to apply product management techniques to to your product You should be looking at some point into defining. What are the key metrics? For it so that's number one I Think the second one and I think this this is a big one for a lot of people is that you as a platform Product manager you have a lot of customers and if you don't have some sort of Quantitative way or quantitative data that kind of buys your decisions You are sometimes drawn into situation of you know, the loudest voice in the room rules And this is just that you know, whoever screams the loudest will will get their feature prioritized Obviously you as a product manager need to avoid that but sometimes when you're talking with High-end leadership around what things should be built in the platform It's sometimes hard to argue against it if you don't have any evidence that that wouldn't be a Good approach and data is often a very good weapon to sort of counter argue these These things so that's what another of the arguments why I felt this is something that it's fairly useful The second one is that And I think sometimes this is undervalued but the quantity of the decisions You're gonna make if you are sort of data informed and I'm sort of intentionally saying data informed because It's very hard to be data-driven when you are in a platform team because most of the data that you will have is fairly discreet, but at least what I'm what I'm Really sure of is that you can be at least data informed which means that you can have a set of metrics that at least advise The direction of where your decision should be going and I think that will you know Immediately improve the decision the quality of the decisions that you make and you will find Sort of how people Realize that the quality of your decisions are much better Fairly soon because not only you're gonna make better decisions You're gonna justify them better because you're gonna have backing evidence on why something is the right thing to build Etc. Etc. Another thing is that And this is something that sometimes you don't expect as I said you usually don't have a lot of customers when you are a platform team but Which means that you can talk with a lot of them and get a clear idea of what's sort of What are the use case that apply to a lot of the customers? But actually if you actually define KPIs and you start tracking them and analyzing them You realize a lot of things that you wouldn't have otherwise found out with just Qualitative data like doing interviews with your customers. There's a ton of things to unlock there For you as a product manager and it's any very valuable source of information that We are living in the table if we're not looking at doing this Then I think you know Those are the key messages there are more but I just wanted to be brief around Kind of the most important things in my mind I think most of the things that I talked about before are Reasons not only to define KPIs in platform products But in any type of product in a B2B B2C like those things apply to To any type of product and as I said a lot of the trending is to start applying product management techniques to platform products and Defining KPIs is just one of the key things that we usually do in the craft Okay, so then you know, hopefully I've convinced you to start doing this and Now we just want to warn you on some of the things or obstacles that I found along the way I think one of the main things that I've found is that I've really touched upon this before but You have a very discreet number of customers sometimes when you're in a platform product And a lot of the arguments against doing this exercise of defining KPIs is people saying, you know We could just talk to our customers. We would learn everything. We just need to ask them and they will tell us You know, which is the most important feature how many times they use these or that And you know, I'm not saying you shouldn't do that. I think that's a great source of information That's what we call qualitative data, but I think it is People often as I said underestimate the value of the information you can get from quantitative data because most of the Information you get from your customers is biased by their own opinions and sometimes they even say things That might not be that true They just think they're true Like for example, you if you ask things like how many times do you use something? You know, that's not a good question to us and and I think those Questions are better answered by data than they are by asking your customers directly The second one and I think this is a big one is that Usually leadership is interested about KPIs when you need to justify why you exist as an as a platform thing and The the biggest pitfall is trying to attach some sort of ROI or money into Your product and I think This is a big pitfall because you will get into a sort of rabbit hole That is endless where you're trying to justify what is the revenue that the platform product is bringing when in fact platform products are just Efficiency-based product, so you're just trying to make everybody faster and by nature you're not bringing any Directly you're not bringing any revenue to the company You're just making sure that everybody can bring in revenue faster than they would if you weren't there. So With that definition is gonna be almost impossible for you to find KPIs that tell you sort of a dollar number on your product I think another one is, you know, you've decided that you want to do you want to define KPIs You think it's a great idea And then you start, you know teaming up with some UX designers engineers What have you and then you start defining what are the KPIs that we want to have for our products and often This conversation drags forever because everybody sort of trying to Find the perfect KPIs and then there are arguments that you know, how are we gonna measure that KPI or you know Could we have some other because I don't think that is that important There's another one that is more important than this one, etc. Etc. And then there is sort of an endless cycle on how to Define those KPIs that it's sort of a dead march because then people is gonna get tired of it And at the end it's not gonna happen. So this is one of the things that you need to Be wary, you know coming in and doing this exercise And then sort of, you know, imagine you go through that pain of defining the KPIs, you know, what you want to track You know how to track it And then sometimes there's Or let's say always you have the problem of okay now I need to prioritize instrumenting my products In order to collect this data Over feature development that I'm doing for my customers and a lot of product managers have problems doing that because in the end There's always a happy customer. You've been the feature whereas if you instrument your product It's not that immediate to understand the value that you might have there So because you're not able to justify why you're going to prioritize that you don't want to do it because otherwise your leadership is going to sort of Ask you why you're taking that decision or your customers will and you as a program manager are not prepared. So this is something that You know, if you really want to do this, you need to overcome this fear And fight against no prioritizing these against other Feature development Uh, and finally, uh, I think another one that I've seen a lot in platform Products is that a lot of the conversations around the kpis are driven by engineers And therefore a lot of the kpis that end up being defined Are what I would call technical performance kpis like, you know, how Is the system stable? How many crashes? What are the number of requests that we have? Etc etc, which you know, I'll go through this later That might be the right metrics for your product in some cases, but in some others They're not And they don't measure the value that you're providing to your customers But it's usually the easiest thing to define, especially if you're working with engineers. So be wary of this because If if you go down the technical performance metrics route It might be that that will not give you any meaningful data for you To advise your decisions as a product manager Uh, another of the big problems that I've seen is that Sometimes people trying to do this exercise without having any analytical skills, uh, which means you don't have any Background on analytics or there's nobody with background in analytics helping you do this exercise Or you've never done this before and the reality is that it's a hard exercise to do by yourself if you don't have any experience and and Also, even if you don't have experience and you try to do it If you don't have a way of validating this with somebody that knows how to go about doing this exercise it will undermine your Authority with leadership or with other people in the team In order to justify that you need to do it Um, okay, so those were just Uh, short list of examples of things that I've encountered myself that I think are important to Uh, take into account going into this exercise that and can really Either slow you down significantly or even put a stop to to to your effort to define your your platform kpis um, so let's just Now go into okay. What are my? My tips and tricks that that were for me when when I was doing this and you will see that most of them are counterparts to the problems that I Said before First is that if you don't see a lot of buying in the organization to do this Uh, I think you need to take time to build your case and what do I mean by that? I mean You need to be able to Understand what you're going to do with it like I don't know maybe building some small mbp presentation on some metrics that you've scrapped together Just by I don't know asking people or or getting to some database yourself and wearing it and so on so just come up with an example and Put it in front of leadership in order to showcase what you could be able to do if you had Uh, this exercise done and you have uh a strong kpis That will usually give better like land The benefit of doing this to your leadership to your product team Whoever it might be so if you see this friction Don't just go and try to push your argument because it's not going to work try to build your case build some Small mbp that you you put together and use it as a sort of ram in order to to get things going Uh, the second one is And I think this is really important is that if you don't know how to do this or you don't have any analytics background Or you don't have any sort of data analyst or somebody that can help you do this exercise Try to find it you know Read a lot You know ping people in the community try to get somebody to peer review what you're defining and and so on because that will Massively increase the quality of the work that you're doing And in turn it will sort of get to results a lot faster Because otherwise you're going to have to learn all these skills at the same time as you're doing a hard exercise to do And that might be uh really troublesome for you. So just find the analytics skills somewhere if you don't have them And then sort of one thing that worked really well for me is that If you don't know where to start in terms of what my kpi's are and this applies to any product really Start with the value proposition of the product the important thing is to Understand what is the value that you're providing to your customers? Like what do they get with the product that i'm providing them? In a platform product and then try to define the metrics around that um so and and and then you'll obviously need to sort of uh tweak things The things here and there because some things are really hard to measure and some others are not that hard But uh start always with the value that you're providing and try to put a measure on that That will You know, hopefully you've already done this exercise, but if you haven't I think that will even be a very useful thing to do regardless of whether you're defining metrics like just understanding the key things That your product provides to the to your customer is something that you know, you as a program manager should do But it's also a very good the starting point in order to then define the metrics um, and I think you know, this is one that probably applies to uh organization or platform organizations that are fairly big but uh Basically when you're trying to do this exercise, um then Collecting all these data like when you've defined all the kpi's you you're collecting all the data And then you need to build reports on top of it. This pipeline is not something easy to maintain so While you're doing it trying to think and if you have somebody helping you from the analytics side Obviously that would be a big plus and you can discuss that with with a person is Try to think how you're then going to scale up this effort because maintaining a pipeline of data collection and then data visualization is Hard there's etls in the middle. There's some reporting that you need to put together or whatever that might be So I think start thinking in the beginning on how you're going to then after you've collected all of this How are you going to be able to sustain and use this? In a in a scalable way, which means you need to look into third parties or even you know If you have an analytics or data warehouse team, you can start talking with them on how they could Be maintaining this uh, and so on and that would mean political discussions as well So make sure that you understand this uh upfront because otherwise you're going to face a situation later on where You've done a lot of work But then you can yield the results because you don't have the manpower to maintain this pipeline Cool, uh, and I think you know related to the sort of endless discussion around defining KPIs I think sort of the the best advice that I can give there and that has worked for me is Just get going start somewhere, you know Define your valid proposition try to define some metrics around it and don't overthink it just Have something that you have relative agreement with your your product team And then start measuring that because most likely like you're never going to find the perfect API from the get-go Most likely you will and and even worse How to measure those KPIs is probably going to require a lot of iterations So just get something going Put some instrumentation in the product And then try to collect the data and analyze it and see what you get And most likely you will need to iterate and iterate and iterate This is the typical approach we take with products and it applies in the same way to to analytics and platform KPIs You're not going to be perfect in the in the in the beginning. So just Embrace that thought and just make sure that that you put the That you get things going quickly I Think another thing that usually People is afraid When they're defining is that, you know, we need to find the perfect KPIs that, you know, will be sustained long term And the reality is that KPIs are linked to your product strategy. So And product strategies change over time, which means that, you know, obviously they don't change every day or every month, but You're trying to achieve something and once you achieve it, then you need to people to define what is the next thing that I'm going to be tackling From my strategy and that might mean that you need to change the KPIs that you have to find And but that is completely fine. So I think you need to make sure you understand that first You if you don't have a product strategy, you probably should have one But you should be linking your value proposition and your KPIs to your product strategy And then make sure you understand that at some point those might change and it's fine There is no problem on on changing KPIs because maybe your people, you know, the the strategy that you are after or You you found out that they're not right as we said before and I think that's completely fine There is nothing wrong with defining KPIs along the way Don't sort of be afraid that people is just going to not trust you I think people understand that in the beginning, you know, you need to go through an iteration process And if you communicate that you're changing strategy, it's a given that then your KPIs most likely will change as well another one and this is really important to sort of back up and build your Confident or the confidence of both your product team and your leadership to to yourself as a product manager is You've justified that you want to define KPIs and you need to put an investment to do so after you Have everything set up leave up to your work use that Thing like you're the customer. This is sort of the dream of every Product manager. You're the customer of what you're building. So make sure that You use it you sort of prioritize it for a reason So once you have it make sure you put the time to look into those metrics Make sure you do analysis and deep dives and obviously if you have an analytics team, they can do it for you But make sure that you leverage All that effort and all that data in order to advise your decisions put them in presentations around, you know, how things are going around the products Use it in product discovery to justify what the best solution might be based on previous usage Use it to even justify whether an investment was was good So if you if you actually try to solve a problem make sure that you have The right metrics define that you can then look in the end to justify that that was a good decision and put that into your sort of standard processes and then And this is a tough one because it's easy easier said than done, but try to avoid Putting a lot of energy over measuring the revenue contribution of your platform thing It's really hard and I've not seen any case where this is successful So try to manage if it's your leadership that is doing this try to manage this Upwards and try to spend the least amount of time possible because these type of Mindsets also drive you towards You know, how would we Calculate this and usually they tend to lead towards lagging indicators or vanity metrics because the reality is that there is no good way to measure this So you find a buy As a somewhat scrappy solution in order to measure it and reality is that that's not going to give you enough information Or if there is it's going to be too late because it's a it's a lagging indicator that happens Very long time after you've delivered or not delivered the value to your customer. So Just be aware of this Okay, uh, finally, uh, some examples, um, again, I don't want this to become sort of the the The list of metrics that you need to find on a platform, but these are examples that have worked for me So just thought I would share that The first one is initially when you're building a product A platform product you need you want your customers to adopt it So I think adoption metrics are really a good one in the beginning And this is usually kind of your first step When you're building something new is to make sure that everybody uses it or adopts it. So, you know, which customers are using your product It's important when you're launching specifically However, there's some cases where like your customers are forced to use your product Which means that these adoption metrics will not be that that useful in those cases Second, uh I think I usually call these use case coverage or variation metrics and what I mean by this is, you know For how many different things are your customers using your product? Are they just using it for one? Niche thing in the sort of set of use cases that they have Is it something that is covering a lot of repeatable use cases across a lot of of your customers? And I think that's really A metric that is really helpful when you're trying to understand whether your product is solving highly repeatable problems And it's sort of very flexible In the sense that customers can use it for a lot of things Um So I think this is uh, this is a good one where uh, you know Your product is something like the value proposition of your product is to be flexible and be able to be used in the in a lot of different scenarios It's also good to understand sort of the penetration of your product across your customers like how deep into their Ways of working is your technology? Because that's that speaks to the stickiness of the product as well A third one is that sometimes platform products one of the key Attributes is to be extensible so Extensibility metrics like if you have ways to extend the the your product It's it's actually sometimes very useful to measure how many extensions your customers are are pulling in place And I think this is another one that sometimes depending on the value proposition of your product can be interesting Uh, and this is I sort of argued against the technical performance metrics But sometimes the key value proposition of a platform product is to just work Be scalable, you know be stable and so on so in those cases actually the kpis are You know the good kpis are technical performance metrics, you know uptime numbers incidents response times, etc, etc Um, so I think you know in some cases they are the right metrics So don't take me wrong on the on the previous argument that they said And then uh, finally, uh another one and this is a tricky one, but uh, when you're trying when your product is fairly mature You sometimes want to optimize, you know, how much time am I saving for my customers? So what I would call workflow metrics are in those cases really useful because they can be The right drivers for you to understand whether you're making an impact on your customer So like what what's the time to do x tasks? And so on The thing with those metrics is that sometimes it's very hard to measure the time it takes so In my case, it's been sometimes useful to flip it around and just calculate how many times they are doing that tasks Instead of measuring how long it takes to do it That has an assumption which is that you know the faster you make The easier you make the task for them to do then they will be doing more of them Which might not be the case because sometimes they're just not gonna do more of that task They're gonna put that time that you're saving them into some other tasks But you know, you need to be aware of these situations and if you uh, if you are in the right, uh Context it might be a good one to measure Cool, uh, that was it for me. Uh, I hope it was uh helpful. Um, as I said I'm really excited to be able to share this with the community and If you have any questions, just make sure you you shoot them my way and I'll I'll try to give you uh any hints, uh, if I can Take care. Bye. Bye