 Hello everyone and welcome to this webinar on how to measure a success of data platforms. Considering data platforms as a product, this session is intended for platforms that are already up and running. And you have been doing everything you can to fulfill the user demand. So now it's time for you to bump up your efforts and think about how successful the platform has been on different fronts. Is it solving the user problem? Is it doing what it is supposed to do? Etc. In this session, I will introduce you to a framework for measuring success. Our focus will be around internal stakeholders. However, there are various measures that can be useful at different stages of products. So let's begin. Before we look into the agenda for today, here is a bit about me. I'm a group product manager at booking.com in data services. I look after a portfolio of different data service products and I'm currently working on building a data foundation layer for booking.com. I live and work in London and my office is in Amsterdam, which is fantastic. In the last more than 12 years of my IT experience, I have predominantly worked on building data and digital products. So here is a list of items we will be discussing today. Definition of data platforms. Why is it important to measure success? What stakeholders are we focusing on? And then I will be introducing you to four measures that define success or the success framework. So putting a definition to data platforms, we are talking about platforms that are meant for acquisition of data, storage, preparation, delivery and even governance, which means security, quality, discoverability of data, et cetera, attributes. And some examples are like big data platforms, customer data platforms, any sort of analytical data warehouses, or even storage platforms like MySQL or not only SQL platforms, et cetera. These platforms can be either on premise or in cloud. So why do we measure success? Well, it's important to see if your investments have been worked and you are achieving cost efficiency. If the platform is helping with business goals or is it helping to make sensible decisions? We also want to measure to get better, so by identifying the weak points. And once you get into a cycle of continuous measurement and improvement, it will bring more opportunities to your team and help with employee motivation. So before we start to think about measures, there is one more important step to take. Thinking about our stakeholders. These are the people who matter to your platform and your measures are influenced by them. Platform users, adopters, business units, these are great, but influencers and sponsors, who are these people? So influencers are people who influence your platform to grow or change like competitions, enterprise architects, even people who are responsible for organization-wide decisions like stopping a particular vendor or supplier or any enterprise solution. Sponsors and turn up people who are directly or indirectly involved to fund the growth of your platform. Given this, sometimes, however, you just might want to focus on one cohort at a time like business units. So that was a bit about stakeholders. Now, I will tell you a bit about my story and my learning that has helped me organize these measures. Then I was working as a product manager for Big Data Platform in one of my previous organizations. I was operating a continuous cycle of asking my users how they feel about the platform. And then I would supplement this information with some system KPIs to conclude what the platform score is at. However, during the course of time, I realized there is a much bigger picture out there. All my stakeholders actually were producing a lot of business value, not just commercial, but several improvements like optimizations of certain supply chain processes. And even though during the requirement gathering phase, this was discussed, there was always a figure explored while experimenting. And the final figure had a completely different picture, which was gone waste. These numbers were big and without the platform, there was no resourceful way of doing the work. So I learned to factor those aspects into my toolkit to make sure they do not slip away. I've also experienced that user feedback was a very important role in the success of your platform. So let's take an example. There is a lot of buzz around working from home and would hybrid be better or complete remote working be better. Is there an easy way to get this data? Rather, no, we need to ask individuals what they prefer to conclude the answer. And in a similar way, I have found asking users to gather qualitative data has been of immense use. Therefore, this framework has both qualitative as well as quantitative measures. So with that thought in mind, let's begin to talk about the actual success framework. This framework consists of measuring success through four different attributes of the platform. One being, is the platform providing value to its users? Second, is it adding value to the organization? Third, is it flexible to change? And finally, when it changes, how do the users adopt or react to the change? Remember that these measures are for evaluating success. So are most beneficial when you have taken some action in these areas. And now you want to see how successful your actions have been. Also, these are just categories. So it's flexible framework. You can pick and choose what applies to your products in what situation. And even when you want to just get started on your platform, success is far away. You want to just pick and choose what you want to implement. You can always refer to this toolkit. Measure one, so let's begin with measure one. And as I said before, each measure has been divided into quantitative and qualitative measures. Let's look at the quantitative measures. I have organized the quantitative measures into two different stages of a product lifecycle. Growth and maturity. So depending on where your product is, you can pick and apply these measures. Growth, for example, here is when your focus is more around growing the platform. It is helpful to look at the number of users, active users, data producers, consumers, etc. When you are at a higher stage of growth, it's good to look at how much you have filled up your platform. And what are the system statistics? This will help you realize when to stop growing, look for further scaling, and transition your focus from growth to maturity. Maturity, this is the area where your investments are low, and you are working towards improving or maturing different features. You're working towards optimizing costs, you're working towards resource optimizations, etc. So the intention is really to get better at what we're doing already in this stage of maturity. So in maturity, we focus on things like SLAs, performance stats, cycle time for onboarding new users, interaction failures, and things like that. And when the platform has grown enough and matured enough, so in the fourth quadrant here, we can start looking at the data attributes of the platform. Like governance, data discoverability, quality, where you can look at accuracy indicators, completeness of data. And this itself is a vast field. So that was quantitative. Let's look at some of the qualitative data. Sometimes all you need to do is just to ask your users how satisfied are you with the platform. Would you recommend it to another colleague? Does it help you solve your problems, etc. Some friendly questions. You can collect these questions through a survey, either through a survey or you have a big user community, or through user interviews if you are focusing on one cohort at a time. Let's look at measure two, which is about verifying if the platform is adding value to your organization. Now, going back to the thought where what I was mentioning about how the platform users are building and user facing products using the platform, it becomes essential to realize how your product will make an impact to the organization, how important an entity it is, and what would not happen without it. In that thought in mind, let's look at the quantitative data or the quantitative measures. In the growth stage here, you worry about costs of acquiring business teams as business teams produce business value, onboarding more and more business teams becomes important so that they build data products using your platform efficiently. So once you have onboarded customer and finance teams, you might care about marketing team or legal team. Once you have grown significantly to business areas, you read your point where the investments are equal to returns and no one is improving any further, which is going to break even points. So you realize the break even point and you measure what figures you have made so far and then you transition to maturity. In maturity, we worry about how much cost has a customer produced in their lifetime using the platform. This is also an indicator for platform that store customer data. And as you move up, you can finally start measuring commercial figures like profit made to the organization and sales indicators like numbers of bookings made products or items sold etc. And finally a big one in this quadrant when you have grown and matured enough. It is how much contribution you have made to the organization's commercial targets like saving operational costs. This is a great measure and a really important one. Qualitative. Finally you can ask your business teams, why do they use our platform? How was the onboarding experience? And if the platform goes down for five minutes, what is the impact on their teams? Measure three. So after focusing on users and business value, this is a really important attribute of a platform. I call it as flexibility. As you know, technology is changing rapidly and it's improving as well. We try to figure out what will it take for your platform to adapt to the changing technology. Can your platform support innovation and how quickly can it transform to start leveraging from the new solutions? For example, for moving to cloud native solutions like Kubernetes, will you have to do a massive change taking months of work? Will the users have to wait a long time and make efforts before they can start to realize the benefits of the change? If the answer is yes, you are in a rigid environment and something needs to change. So let's look at what these measures define. Now, innovation is happening in the organization and we determine how well can your platform support innovation? So during growth, we look at new data assets or new data products created and added to your platform. And we also keep a record of the number of new enterprise applications that have been onboarded to the platform. This indicates how easy it is to have the application plugins and various integrations present on the platform. So these applications can get connected quickly and easily. Once you move up in growth, you focus more on sophisticated technologies like artificial intelligence and machine learning. And once you transition to maturity, automations are an important measure. So what are the requests for automations or if the automations you have realized versus how many you've achieved so far? Also, how quickly can you move to cloud native technologies? And the final indicator here is the number of green field applications, which means new applications created from scratch using latest technologies using your platform. And again, this indicates how well innovation is supported. And finally, we also record observations on behavioral changes. Like if you cannot move to a new technology, how does it impact your users? Do they move out? Do they create something of their own or do they stop using the platform or do they use it a little less now? So moving to questions to ask people. Can they innovate using our platform? Is it easier to create something new and also what prompts them to look at other options? These are some useful questions to understand how do the users react to the flexibility of the platform. And the last one, and this is tricky to understand, but it is equally important. You've changed something like adding new capability, new feature to your platform. What is the user reaction? What is the user satisfaction, adoption to this new capability? How easily or quickly can the users start onboarding to these changes? Maybe measure how many users have welcomed this change. So this is about this measure, but why this measure? Why is it important and why is it in consideration? Many times I have seen a new capability was requested by a group of users with the idea that more and more users can benefit from it. However, I have found that the other teams were quite reluctant to use this change because they need to learn the technology, they don't realize the benefit from this change and the cycle time to get them started was too high. So I took some actions to solve this problem, like providing an effective communication about changes that highlights benefits, conducting some trainings, giving demo to a group of people and also optimizing the cycle time. And now it is time for me to measure how successful those actions were. So I look at quantitative data. So during the growth phase, you look at the number of capabilities you added and number of processes that now run on the new software. And further up in growth, you worry about the adoption to the new capability. So the number of users who are using the new capability, the utilization trends, users trends and number of users added as a result of the new capability. People who have participated in the training, that's a fantastic way to assess how and what is the interest of people in the new feature. During maturity, we looked at cycle time on onboarding users. And further up in maturity, we look at investment versus results. So things like cost reduced as a result of the new capability and productivity improvements like ours saved. So these are some quantitative indications about the attribute adoption to changes. Let's look at the qualitative ones. So I'm going to ask my users here, how well can you navigate through the changes? I'll ask feedback on documentation, communication, and how do you feel about the new capability? Does it help them do anything better, et cetera? So as you can see, users have been constantly involved in the growth and maturity of the entire process. So this brings us to the end of the session. And here is a closing note from me. So other than success measures, you have various functional and non-functional measures that you would worry about for your platform. What is performance, growth, sustainability, scalability, elasticity, reliability, et cetera, et cetera. And you as the champion of your platform can decide what to measure and at what stages. What will be the north star and obviously keeping users and value at the heart of all measures. I really enjoyed this session and I really want to thank you guys for joining me and hope you had a useful half an hour. Hope you enjoyed and gained something from this session and good luck measuring. Thank you.