 Hello, everyone. It's not what you think it is, indeed. This is Max Verstappen, Formula One driver, three times world champion Red Bull Racing team. And to me, Max Verstappen represents the most precious assets of a tech company. Engineers. Engineers have the unique ability in tech organizations to create so-called 10x impact with their ability, with their work. Quite the task, quite the expectation, right? They need help. Meet Hanna Schmitz. Hanna Schmitz is a key person behind the success of the Red Bull Racing team. By the way, Hanna is a real person. Please look her up, Hanna Schmitz. I couldn't find any royalty-free image of her on the internet, so Jen AI helped out here. Now, she's the principal strategist of the Red Bull Racing team. She's responsible for Max on risky takeover moves, on understanding how the weather might influence the race. She is responsible for building an entire management strategy. In other words, she needs to make sure that Max wins. Now, to me, Hanna right here very much represents this crowd right here. Just like Hanna, product managers like you and myself, we need to make sure that the Maxes in our organization, our impact is maximized. Today, we're going to talk about this dynamic, dynamic between Max and Hanna in our organizations in the setting of tech companies. I'm going to zoom in on this relationship between product and engineering. I look very much forward to sharing some of my personal stories and trying to get the most out of this unique relationship. But like with any relationship, right, there is unfortunately no silver bullet to success. No silver bullet to success. Instead, I hope that my talk contains some elements that are recognizable in your organization, maybe actionable, and who knows, maybe the relationship gets much better. This is the context in which we needed relationship counseling. Today, I'll introduce you to the complex world of payments. We're going to discuss the flow and how a payment flows for our payments platform. And I'll hopefully be able to show you what tremendous value machine learning can unlock in this particular setting. And this is my role at the company called Agen. My teams and I, we are responsible for building products that leverage machine learning to provide shoppers like you and I with a seamless journey when paying. Well, we also have to make sure that our customers, merchants, are protected from fraud and other risks. So Agen is a financial technology platform. We provide payments on a global scale in online environments but also offline or with physical terminals. And besides that, in the realm of financial services, we build many more products, think cash advances, loans, business bank accounts, cards, you name it. And what's very cool about financial products is that data is either the perfect fuel for skill or the perfect optimization layer. And to give you a sense of that skill, last year we processed over $1 trillion for our platform, for our merchants. These merchants, they have expectations. They expect that shoppers like you and I, that we have this fast, happy, beautiful, uninterrupted flow when paying at one of their stores. Now unfortunately, this doesn't always go according to plan. Maybe recognizable even to some of you. 9% of good shoppers, they can check out. They experience inefficiencies in the payment flow. This can be the fact that you can select your local payment method, your favorite local payment method. Or you might get flagged accidentally or classified as a malicious user as a fraudster blocking you from transacting. Or you might get caught up in some legacy that unfortunately very much still exists in the complex global infrastructure in the financial world. This is what my team has now tried to solve for on a daily basis. Now this is the payment flow from left to right. On the very left you find this checkout box. This is the UI, the environment in which you select your favorite payment method. You leave some personal details. And from there we activate the risk checks, the risk box right there. In the risk box we need to determine, are we dealing right here with a malicious attempt with a fraudster? Fraud has always been a very complicated problem in the financial industry. But especially now thanks to the rise of global e-commerce, the advancements in GenAI as Joe pointed out, and these advancements but also the fact that you can now open up bank accounts in a matter of minutes online. You don't have to show up at a bank anymore physically. That made this a very hard problem to solve for. Then we have the authentication layer that applies to some payments. And in the authentication layer I'm referring to technologies that try and determine is the user of this particular payment instrument, for example a card. Is it also the owner of that payment method? So here for example Face ID on the iPhone at SMS tags when you're buying your flight ticket. Those are authentication mechanisms that we use very much linked to this fraud and risk box too. Now if everything is all right in those boxes we move to the first attempt stage. So we built up this massive transaction message based on all the information we've derived from all these steps but also business model information, the channel, all the context, all the technical context we can find and we sent this to one of the networks to Visa, MasterCard, American Express, local payment method. And we're hoping that these networks forwarded correctly to your bank, the bank that gave you your card. And we're hoping for an authorization response. With this authorization we can tell the merchant you're good to go, please provide the goods and services. Now unfortunately this doesn't always go according to planets we've seen before. There may be insufficient balance on the bank account. It might be the case that your bank still thinks it's fishy despite our checks. Or maybe there's some network errors as we pointed out before. Now the teams that I work with we're all trying to make sure that we make the right decision, the right complex decision in every box for every individual payment that flows for a system. What's very hard about this problem is that this needs to happen in a so-called low latency environment in real time. When you're booking your taxi with your favorite ride-hailing app at midnight or paying for your groceries, you and I as shoppers have the expectation that this goes blazingly fast. Also all these teams, yeah there we are, all these teams they're trying to optimize in their particular setting, right? In the checkout box the team is trying to leverage UI to play around with the UI to notch you into conversion. Well in the risk box it's all about finding that equilibrium between risking fraud versus risking blocking a good shopper. In the second attempt stage it's all about calculating the probability of a successful retry, right? We can maybe play around with transaction messages I've been talking about maybe we can get a successful authorization but you also have to take into account dimensions such as additional payment cost for example. Now to solve for all of this complexity at an enormous scale we've introduced machine learning algorithms over the years, massive investment from our side and you can imagine with all the fraud cases that we've seen in the past, right? All this labeled data but also the preferences from all the banks in the world we use all that information right now together with all the real-time events flowing for a system to make the right decision at each particular stage in the payment funnel. Now to give you a sense of skill I'm talking about more than a thousand requests per second going through this particular part of our payments platform. We already, I shared something around quickness, right? All these models they need to come to their conclusions in less than 100 milliseconds and this accumulates to over 300 billion requests per year flowing through this particular part of the payment platform but the investment it's all worth it, you know? Referring to the trillion dollar amounts basis points count in this business case and by the way I also think this is just a very cool ML like a very cool problem to solve with ML you can imagine. So now we've established a business case I hope I shared the behind machine learning in this particular setting and I would like to go back to the relationship between product and engineering the dynamic between HANA and MAX. Because you can imagine that in order to solve for all this complexity we need those supermaxes, right? We need those top engineers that solve for this and they need to be guided by HANA. What I'll do I'll share some manifestations of how this went wrong on our side actually and hopefully I'll also be able to share some fixes with you so that we can have a better relationship after some proper counseling to now benefit from a better relationship and make sure that the maxes in our environment have more impact. First up goals and prioritization now a while ago this was our situation all the teams working on this payment flow optimization they were represented in our objectives and key results goal-setting methodology also according to textbook metric that all teams hopefully can link towards to and it perfectly represents the value that we create for customers and everyone happy. Unfortunately we chose a very complex north star metric now engineers got frustrated they got caught up in their own complexity much more focused on perfect measurements rather than quick iterations not getting the guidance from product managers in needs isolated teams, isolated individuals isolated features with so many OKRs and a complex north star metric engineering teams, product teams started very much working in isolation using local success metrics today we only work with three OKRs and the key to this has been prioritization. I'm sure you talk a lot about prioritization. Now a classic way of putting this is that prioritization must truly hurt and this might sound easy but in my experience and I think this is only human we want to make sure that every previous investment every product every project every initiative somehow ends up on planning we want to make sure that every individual every team every role every function can identify itself in KPIs or in OKRs don't prioritization must hurt truly the good news is that your engineers they will love you for the fact that you're doing this ruthless prioritization you will reduce complexity and as a result these engineering teams will become so much easier to measure success and it will become so much easier for an engineer to understand how they contribute to the big picture product first thinking second up, ways of working again, previous situation we had in the teams working on this particular problem domain in this problem domain we had a ratio of 1pm for every 6 engineers and also we were falling for the trap where pms get caught up in way too much project management and stakeholder management work the problem about this is that this creates high coordination costs within your organization and trust me the maxes of this world they don't like meetings we don't like meetings imagine engineers fluff work, unrelated decks flying around this doesn't help and it only creates isolated thinking isolated features today we're looking at a 1 to 8 pms to engineers ratio and I personally think we should go even more up maybe to 1 to 10 now we've given product managers much more scope much more the great thing is that this scope naturally created product first thinking let me give you an example in the previous setup we had 4 pms looking after 4 different models and when you ask smart people to go for something they'll find ways to identify new use cases, additional value more research, more applications but again this results in for disjointed features today we have 1pm looking after 4 models these 4 models and all the work that's being put in in one particular model hypothesis testing, monitoring systems you'll name it all that work is now being extrapolated to the rest of the models and not only are engineers happy that there is just one person to interface with it also naturally created a guardian of the story of these 4 models this pms is now much better equipped much more able to translate all this ML complexity to the commercial organization also we did some proper counseling pms in the room together and we decided let's shift ticket assignments, print planning all that project management stuff let's shift it to the tech leads not the pms anymore for us this worked tech leads feel much more in control they can much better anticipate the pressure in the workloads on their teams all the pms are now freed up to do meaningful pms work writing customer inspired PRDs and road maps and last but not least product first thinking allows you to land a product instead of launching a product let me explain this with an analogy I have from a colleague actually when you are launching a rocket lift off of the ground that doesn't define success you're trying to land it connected to the international space station landing it on the moon landing on the Mars and the point I want to make is that only with product first thinking you'll be able to set out a logical steps a logical plan for the go to market only with product first thinking you can optimize the product operations that will definitely follow a launch when you launch isolated features all you do is increase coordination work and you'll create massive unplanable overheads for engineers and then we have impact an impact I can best illustrate with an example I think because something very interesting happens when we applied product first thinking in this particular domain there was an unlocked value that was always right in front of us all we had to do is apply this way of thinking remember the payment flow we discussed before together I copied over three particular stages right here and we've determined now together that these local success metrics this isolated features thinking that's not the way to go and don't get me wrong I'm sure ML teams they'll always have their own statistics their own ways to evaluate impact but there needs to be some global success metric a simple one that interconnects all of this complexity in our case that has become the full funnel conversion rate it's a simple metric all these ML models can understand, can build for and moreover very important this is a metric that's also very close to the customer world this is a metric that they'll understand even better than us sometimes so now we have this global success metric we have fixed the relationship between product and engineering right we've applied all these fixes in ways of working and prioritization and we can stop thinking about this payment flow in sequential steps because remember when I explained the product flow the funnel, the payment funnel I went through it black box for black box black box let's look at it holistically right now as a whole product-first thinking it turns out that when we use the information the output of the risk model as input for the authentication model that's a total game-changer transactions where the risk model is not sure what to do and this happens right you're not sure if it's fraught you're not sure if it's a good shopper we now use the authentication layer dynamically using ML much more intelligently to try and create extra assurance well in the isolated setup the risk model by itself in isolation might have thought you know what I'm going to be on the safe side I'm going to block this transaction risking blocking good shoppers also we use the conclusion of the authentication model right now and apply that conclusion in the first attempt stage when you have an authenticated payment that's the most clean payment there is guaranteed right phase ID sure there can be some fraud but very little phase ID is a very strong mechanism that means that in the first attempt stage we can be extremely aggressive with all the risky optimizations that are available in my world and lastly using the risk informed data points that our risk model generated even though we know it's a good customer it's any of you just transacting using that and making it available to the partner banks we're connecting in the first attempt stage it increased that full conversion rate simple wins so on the show here that by getting rid of this isolated feature thinking we unlocked value that was always right here in front of us it might look simple but from a technical perspective we were not set up for this success and also from an organizational structure the dynamic the relationship between Max and Hannah and the way we've organized our product team this was not achievable and now it is in that previous setup in that previous setup we were thinking in isolated ML problems and these models in that same funnel they might show greedy behavior they might not care too much what happens down the funnel our teams were using local success metrics well they were all working in the same problem domain not the right approach even worse and frustratingly I think everyone involved experienced something very fragmented not understanding how their piece of the puzzle we're talking about very large teams not understanding how they contribute to the big picture worse is that it becomes very hard to translate all of this complexity to the commercial organization well it's so valuable the commercial organization needs to be equipped with simple ways of explaining all this complexity that we've just discussed today we have a unified offering and by embracing product first thinking we got rid of this lots of complexity right we got rid of what we had in our organization sometimes hyper focus on very particular optimizations in the payment flow only distracting and taking up way too many resources and we unlocked value that we had there in front of us and not only have our customers now an increase in this full vulnerable conversion rate right in value that we eventually have to create for them they'll experience this too remember the black sequential boxes in our previous setup we had in our customer area for each black box a way to explain that part of the story because every part of the story had its own success metrics, its own statistics again a fragmented experience for customers I think you've guessed it right today very simple we're trying to bring this all together into one view one simple way to demonstrate that top value and not only is this way easier for customers to interpret instead of trying to bring all these stories together how does risk influence first attempt all that stuff is now just much easier bringing them back together and besides customers engineers love this too as an engineer to understand how do I contribute not only to the big picture but to the customer impact and not just to my individual black box so while it might look impossible or very very hard or challenging and this can be due to your culture size of your organization existing complexity in your product suite ways of working caught up engineers I hope that my story that it provides actual evidence that you are able to reduce complexity it is possible in a very large organization to go back from 21 OKRs to 3 OKRs do get in a room together with the maxes in your organization which are engineers and discuss is there project management work that we can shift back to the engineering team simply because it comes very natural to them you save coordination costs and meetings we can't give product managers much more scope trust me they'll create naturally this product first thinking benefits the customer and it is possible to hook up 3 ML models together while it might look very difficult right in this demanding low latency environment it is possible if you embrace product first thinking so just like Hannah get together put your heads together with the maxes in your organization tomorrow product is a team sport challenge complexity together get rid of isolated features thinking and start maximizing your impacts with product first thinking thank you very much for your attention I wish you all the very best on your product journey